This manuscript explores the intricacies of researching within one's own culture and addresses conflictual and concordant issues that arose when an African-American woman researcher interviewed other African-American women. It is set forth that there is no monolithic group called African-American women and that tensions arose in the interview process relating to the differences in color and class. While the issues of race and gender were unifying issues, the participants often conceptualized them differently. The data used herein were situated in a larger study; the purpose of that study was to examine the educational narratives of reentry Black women in an effort to determine how the dynamics of the larger society, which often negatively impact their lives, were played out in higher education. The sample consisted of graduate and undergraduate African-American women between the ages of 34 and 54. This qualitative research project used narrative analysis as the methodological approach and Black feminist thought provided the theoretical framework.

Introduction

Within qualitative research there is a significant body of feminist literature that addresses the research dynamics that occur when women interview women (Cotterill, 1992; Minister, 1991; Ribbens, 1989).[1] It is generally set forth in these writings that the common bond of gender provides the groundwork on which to construct trust and dialogue. Similarly, it is assumed that when Blacks interview other Blacks an empathetic understanding will be accorded across racial lines. And the synergistic extrapolation of each of these conclusions would be that when Black women interview other Black women there exists an immediate perceptive bond of sisterhood that provides an ideal research setting (Beoku-Betts, 1994; Etter-Lewis & Foster; 1996; Foster, 1996; Nelson, 1996; Vaz, 1997). However, all three deductions discount the intersections of societal barriers omnipresent in a hierarchial world. Certainly, class concerns can cut through gender and racial solidarity and concepts of gender and racial identity can vary among women.

In a recent qualitative research project in which the purpose was to examine the educational narratives of reentry Black women (Johnson-Bailey, 1994), the issues of race, gender, class, and color emerged as themes in the schooling stories of the women being studied. However, these same dynamics also surfaced as factors that affected the research process. There were instances where the Black women being interviewed assumed that the interviewer's status as a Black and a woman assured not only understanding but also empathy on race and gender issues. Yet, there were clearly moments of awkwardness during the interviews when perceived class and color differences hampered the flow of conversation. By examining these themes in the larger qualitative study and the research process, this article explores how race and gender brought me, as the researcher, closer to the participants and how color differences and perceived class differences forced us to struggle.

The study, in which the researcher was also a participant, used narrative, biographic, and linguistic analyses as methodological tools (Alexander, 1988; Denzin, 1989; Etter-Lewis, 1991) to probe the lives or a group of graduate and undergraduate Black women attending schools in the Southeastern United States in an attempt to discern what themes emerged relative to their reentry educational experiences.[2] The women, Sheila, Marcie, Lynda, Cathy, Beth, Jean, Faye, and Juanita (all pseudonyms except for the author, Juanita), ranged in age from 34 to 54. Their accounts represented energetic culture-bound stories of battles within an unaccepting and hostile academic environment.

Between us women: bridges called race and gender

All of the women in the study possessed an understanding of societal hierarchial forces that shaped and determined their existences. They identified racism as the specific dominating factor and they used an oppositional world-view to frame their stories. Using what DuBois (1903/1953) called "double consciousness" and what Collins (1990) called an "outsider-within perspective," the women seemed always aware or a White audience.

Gender, while not advanced by the women in this study as a restraint to their lives or educational endeavors, was observed to be a factor that consistently affected their lives. This was most apparent around issues of self-esteem, childcare, household chores, and relationships. During the interviews, the women in this study communicated gender-bound assumptive connections when they discussed children, husbands, household responsibilities, doubts, and fears.

It is my position that the issues of race and gender were uniting forces. As demonstrated throughout, the women and I established an ease when discussing race and gender. Our concepts of and our experiences with race and gender were similar. Additionally, there was no lack of trust expressed by the women regarding my positionality

relative to race and gender issues.

Race

Race was never raised as an issue in the interview questions. Yet each woman advanced the matter of race, not only in describing her life, but also in discussing difficulties in her educational history. As Brody and Witherell with Donald and Lundblad (1991) set forth," When given permission to use personal narratives to discover and reorganize the stories of their lives, adults will invariably explore themes of gender and culture" (p. 267). Race and the knowledge of living in a race-conscious society was a factor that the participants and I shared.[3] Two areas of unanimous commonality for the women were accounts of painful classroom episodes (which were interpreted as racist) and childhood understandings of racial difference.

Two women in the study, Lynda and Cathy, could recall the exact moments when they discerned that their blackness was perceived as inferior. When asked to describe her most unpleasant memory of school, Lynda, a 54-year-old student at a small community college, cried as she told of an elementary school incident:

When I was on my way to school. And I rode the city bus. And at the time I did not know we were not suppose to sit by White students- nor did the little girl know that she wasn't suppose to sit by me. And we rode all the way to Broadway and Cherry. And the bus driver -- at that time they had those radios, walkie talkies and they radioed in for the cops to meet us when we got ready to exchange buses. ... And I started crying and asking, "Why was I going to the city hall? What had I done wrong?" ... And she [a teacher] just hugged me and said to me, "You didn't do anything wrong. All you did was ride the bus to school." And somehow or another the little girl's parents and the Whites on the bus made a big issue out of it ... And after that experience it was inside of me as a child ...

This event was for Lynda, the oldest of the respondents, a significant marker (Denzin, 1989) in that it forever changed the way she saw White people and the way in which she related to them. Her most painful memory of school was a tale of how she became aware of being Black in the South.

Each of the study's participants realized in elementary school, and some prior to formal schooling, that being "Black" meant being different and apart from most of society. For most of the respondents the cognizance of the difference was further emphasized in the school setting. For example Faye, a 43-year-old graduate student at a large Southern university, remembered integrating an all-girls public high school:

There were only ten [Blacks] in a high school of four hundred ... Definitely minority. Okay. So the whole thing was mind boggling ... We were 4.0 students until we went there ... like the teacher ... had little help groups that always excluded you. That's how blatant it was.

The women in the study shared their accounts of " knowing" about racial difference and assumed that, as a child who came of age in a segregated America, I also knew. In my own narrative I paralleled the participants' experiences by recounting an early awareness of race. During my grade school years, I realized that not only was the Catholic school I attended segregated but it was 17 miles away in a neighboring state. Every morning my sister and I would have to wake up at 5: 00 a.m. in order to catch a 6:00 a.m. school bus. Just 3 miles away from our house was another Catholic school, but it was all-White. Many years later when I was allowed to attend the closer and newly integrated Catholic school, somehow "I always felt as if I didn't belong."

The topic of race continues to stand apart as a force that shaped the women's lives and cemented an early awareness of race. This is demonstrated in their frequent descriptions of their environments as segregated, all-Black, or integrated. Based on the phrasing and use of childhood quotes in their memories, it appears that segregation was such a confining descriptor for their lives that their awareness is not an adult's awareness, realized only in the retelling or reliving of the epiphany (Denzin, 1989), but is instead an awareness that was present in their childhood. It is an understanding of race, albeit through different means and at different ages, that unites the Black women studied and provides a common ground of understanding and analysis that benefited me as a researcher who shared the same racial background.

Gender

While the women named racism directly as a constraining factor in their lives, gender went unnamed. The participants did, however, paint pictures of existences configured by gender subordination. In the educational narratives studied, gender was not generally seen as a separate issue for discussion. Race was most often advanced as the barrier to be overcome. Although on occasion they identified themselves as "Black" in a world that was not and emphasized their race as the most important factor, the participants always spoke foremost of themselves as "Black women" never as women only or as Black only, and the women always classified others by race and gender accordingly.

There were several main areas of similarity that linked the participant and researcher narratives: (1) self-esteem, (2) self-doubt, (3) guilt concerning time spent away from the family, (4) controversy over household duties, and (5) childcare concerns. Low self-esteem is often cited as a psychological barrier specific to nontraditional women students (Pitts, 1992; Salman, 1988), and it is also cited in feminist literature (Boyd, 1993; Steinem, 1992) as a difficulty for women students in general. Cathy discussed her feeling of inadequacy when she talked of her decision to return to school. She states:

I felt an inferiority type complex ... I was afraid. "It's been too long. I won't fit ..." And the counselor will tell you, ah the many times I called her at home and I said, "Uh, uh uh, I can't do this." And she would say, "Yes you can!"

Cathy and I discussed our feelings of" not being good enough" several times during the interview. The dilemma of self-esteem was echoed throughout many of the collected narratives. Lynda, who actually registered for school in 1976 but waited until 1990 before she garnered the courage to actually register and attend classes, voiced similar feelings.

Other defining gender issues shared and understood by the participants and I were childcare and household responsibilities. Sheila, a mother with small children and a husband, referred to her children and household duties throughout the interview. Early in this recent school experience she was forced to ask her spouse for help because her morning routine was making her late for work:

Up at seven. At one time I was getting the baby dressed. I would find myself being late ... So right after Christmas I started getting myself dressed and letting my husband get the baby dressed. So I feel better when I go to work on time ... Sometimes when I leave work – sometimes I'll pick up my daughter and take her to piano practice ... And on those days that she doesn't go to piano practice I get off work and go pick up the baby and have her home about 4:15 and then my class starts at 4:30. So it's rush, rush, rush.

Although Sheila's husband has taken over one of her duties, dressing the youngest child for daycare, she still finds herself bound to her role as homemaker.

Like Sheila, Beth struggles with household and childcare duties. She has also found it necessary to make adjustments in her previous obligations: ... I used to always have dinner on the table and have his clothes washed and have them all folded up and in his drawers. Since I came to school, it's like I think you know how to operate the washing machine.

The issue of housework and childcare and the unequal division of labor, with the majority of the responsibility falling to women irrespective of their workload or school obligations, is endemic to the way our society has conceptualized the female role. This unbalanced workload is a primary ingredient of gender subordination. All of the women in the study felt that school had affected their relationships with their children, spouses, or significant others. It was a shared issue of womanhood that the respondents and I spoke of in synchrony.

Although the women did not explicitly describe gender as a defining factor, several of them alluded to their lives as Black women as being a different and a more difficult circumstance of existence. Marcie framed it succinctly, "I'm a Black woman in America and that's as handicapped as you can get." Cathy's analysis of the situation also indicated an awareness that hers was a different existence: "... Black people, especially Black women -- we have to be 105% qualified anyway -- you know in order to succeed long term." And Jean, a 53-year-old widowed graduate student, felt that the struggles of Black women had

served to make them stronger: I do feel that the Black woman is strong ... Because of the trials and times that we have had to endure over time and ages. And to be accepted we've got to ... work twice as hard ... Time has brought on its share of pain, endurance, and just the need to survive.

These three quotes from Jean, Cathy, and Marcie seem to indicate an understanding of the intersection of gender and race as unique to the lives of Black women and in addition reveal a common conception of Black women as super-strong women.[4] Yet, the women did not present what they perceived as an intrinsic unfairness in the lives of Black women as a gender issue but rather presented being Black and a woman as conditions woven so tightly together that they could not separate them, nor did they see a need to separate them.

Overall, while gender is not mentioned explicitly by the women as a dominant force in their lives, their narratives reveal that gender is a factor that eclipses their accounts, casting a shadow that has affected them in several ways. Many of the women suffered with concerns of self-esteem, doubting their ability to succeed in school or even their right to be in school. Others struggled with problems with childcare and household duties. It was interesting that while race was seen as an early factor in the narratives, gender was visible only in the stories we exchanged about our husbands, children, and life choices. The talk that touched on gender issues flowed easily between us. An understanding was assumed. Gender was a tie of trust present in the interview setting.

Between us women: the schisms of class and color

Just as race and gender can provide a communal ground for empathetic understanding and trust when Black women interview Black women (Beoku-Betts, 1994), class and color can prove to be impediments to the process of understanding or acceptance between Black women (Johnson-Bailey, 1994). I never envisioned as a researcher/participant that my perceived class status or skin color would trespass into the study. But the women in this study correctly assessed the shared bond of sisterhood between us and my conversational style of interviewing as leeway to a reciprocal question-and-interview process (Collins, 1989). For the most part they did not allow areas of class and color to remain unacknowledged or unknown.

Throughout much of the feminist literature, discussions present all women as the same, a norm, and all people of color as similar. When a specific racial group is discussed, it is presented monolithically. The dilemmas that transpired in the field during this research proved such conclusions imprudent. My educational level and attached class status were mentioned by several of the respondents as factors that set me apart as different. It was conjectured by several respondents that "educated Blacks" think of themselves as "better" or superior to other members of the race with the use of statements like, "Well you know how we [Blacks] get when we get a little education," or "I'm surprised you can be so real since you are down there with those people [Whites] at the university."

In addition to my educational level and its attached class level, skin color discrimination was referred to by three of the women in the study as a problem that affected their early schooling and their lives in general. Using direct and indirect methods, several of the women in the study queried me about my opinions and experiences relative to my skin color and hair. Beth, who was discriminated against by other Blacks because of her dark skin color, asked me directly if my lighter skin made my early schooling a pleasant experience.

The times when class and color concerns occurred and were discussed in the interview process proved the most uncomfortable moments for me and for the interviewees. This assessment is based on the long awkward silences, averted glances, raised and lowered voices, tears shed, and harsh confrontational statements that transpired at these times. Class and color emerged in the interview process as barriers to the intimacy and the trust of the procedure.

Class

Disproportionately more Black women and their children exist below and slightly above the poverty level than any other segment of our society (Williams, 1988; Hacker, 1992). As a result, class is inextricably tied to the situations of Black women and their families. The living conditions of the women in this qualitative research project were never directly spoken of by me or by the respondents. Yet the issue of class became an inevitable component in the investigative process through many means, interviewing in the home setting or by listening to the circumstances of their stories, especially their educational goals and those of their family of origin.

The discussion of class was immediately evident in my interviews with Lynda and Cathy. It was necessary to transport both of them to and from their homes since neither of them had a car. During our drive, both commented either directly or indirectly on my "nice car" and asked me questions about my neighborhood and educational background. It was my level of education that seemed to interest them more than any other facet of my life. Indeed, questions about why and how a Black woman came to work on a doctorate comprised the line of questioning pursued by all of the women during the informal conversation prior to or subsequent to our taped interview.

Such talk was later analyzed as pertaining to class. While all the women asked a variety of questions about my family of origin, my childhood, my child, and other usual conversational pleasantries, the only area of my personal life that each respondent questioned in detail was my educational background. It appeared that the women seemed to determine that I was middle class since I had the leisure time, inclination, and know-how (not knowledge) to attempt a doctoral program at a "White" institution. One comment made by Sheila in the final moments of our interview summed up her assessment of our different paths in life: "I'm surprised you're so down to earth. I've enjoyed this." When asked to explain her comment, she stated that most people with such high educational aspirations treat those without the same credentials or goals with contempt.

In terms of how life conditions were manifested in the education narratives of the women, the picture is a complex one. The question of how or if education would be a future class determinant or exactly how it fit into the various women's career dreams was an unexpressed but ever present inquiry of mine. For each woman, life held its own indicators of class and each story its own dreams or desires about class aspirations.

Cathy grew up an only child in a poor, single-parent home. She describes her early life as one of deprivation. She is the most socio-economically disadvantaged of the respondents and lives in a housing project with her two children, a 5-year-old girl and an 11-year-old boy. She receives public assistance and attends a community college on a Department of Family and Children Services grant. Although her home is neat, it is sparsely furnished and conveys her financial situation. Her economic conditions often intrude on her college aspirations, as was the case during the first week of her new school life: I remember one morning I left home. I had missed the bus 'cause the bus would come so early in the morning. I had to get up -- the children and everything. I just missed the bus. I was determined to get out there. And my class was like at 10:00 and I mis-timed it ... So I decided to walk. I said from where I live to the college is no problem until I started walking. And it was raining and II] fell in some water. And I looked back. I could see my house. And I looked the other way and I couldn't see the college. But I was determined to go. I had to go.

This journey of approximately 8 miles is one that Cathy said she had to make because: "I didn't want to miss, not just cause I had to walk. I wasn't sick. The children weren't sick, and I didn't want to miss it. I was determined to get there."

Cathy's determination has served her well. She has not been able to repair her car and now catches the bus to school every morning. She walks her children to their schools and then walks the half-mile to the bus stop. According to her, the nursing degree is her way out of her present life circumstances.

Marcie, the youngest participant, was an officer's child. She knew the privilege that her father's rank of lieutenant colonel afforded and had an early awareness that not all Black children lived in the same kind of Army-base housing. Like Marcie, Jean and Faye also grew up in privileged homes. Jean was the daughter of the local mortician and the school board president. Faye's father was a prominent local minister. Both their mothers were homemakers. For the most part Jean and Faye did not talk directly about class in their narratives or give any examples of how class affected their lives because they seemed insulated from the concerns that plagued respondents like Cathy. Moreover, since class as a matter in Black communities is associated with one's education as well as one's position in the community (Brown, 1991; Frazier, 1962), it follows that the respondents whose parents were educated would have garnered for their offspring a higher social level in the community.

The manner in which Marcie, Jean, and Faye, the participants more privileged by birth and present circumstances, approached the interview was noticeably different from that of the others. Their level of comfort was higher and indeed Marcie took full control of the interview. The level of exchange with these three respondents was never forced and yet they challenged my statements on several occasions. This indicated that they regarded me as a peer and seemed to give no special credence to my vantage as the interviewer.

Generally the lives of the women in this study contrasted greatly. These women represented several social classes: socio-economically disadvantaged, working poor, middle class, and upper middle class. Several of the women were born into their circumstances and others, particularly the upper middle class participants, were assisted into their social classes by privileged beginnings.

So class boundaries, like gender restrictions, seemed undeniably linked in the lives of the eight Black reentry women in this study. They all knew and had experienced either personally or through their communities that poverty was a reality in the lives of many Blacks. While their lives might have been similarly constrained by race and gender, they now live different existences as indicated by their class status.

Class intruded into the research process in various guises. Unlike our shared and understood stories of race and gender, class was not a concern that was easily discussed. When Beth, Cathy, Lynda, and Sheila told me the circumstances of growing up poor and I related my similar situation of growing up two blocks from the city dump, my explanations were not taken at face value as were my gender and race stories. The women responded to my stories with skeptical remarks such as, "Well look at you now," or "Well you wouldn't know it to look at you." So, unlike gender and race, class did not give us common ground on which an automatic understanding was assumed. While some of the participants shared educational backgrounds and goals similar to mine, the circumstances of our families of origin were dissimilar. Those who had been privileged by growing up with middle-class existences stressed how sheltered and comfortable their lives had been. At such times when our commonalities stood as our most recent experiences and should have been points of connections in our conversations, they were instead points of division. In several ways, class circumstances presented walls in the research process that we either peered over or attempted to break through.

Color

As an issue of concern amongst Blacks, race is openly acknowledged as a problem that affects the lives of Black people. It is examined and debated in Black communities. However, the matter of skin color and skin-color discrimination are subjects that are spoken of less frequently and in a less than open manner.[5] This intraracial discrimination among Blacks gives preferential treatment to its members who have lighter skin shades, thin facial features, and straight hair texture.[6]

Colorism is a vestige from slavery, much as class is a function of a hierarchical capitalistic society, and sexism is evidentiary of a patriarchal system. Presently in the Black community the signs of the importance of skin color remain unmistakable. Skin tones remain a prominent descriptor when Blacks describe each other. In addition, there are linguistic indications of the importance. Language demonstrates how entrenched the notion is by the many labels use to designate shades. Examples such as light-skinned, brown-skinned, dark-skinned, fair, hi yella, red boned, pecan tan, and numerous others are still part of the Black community's private talk. In this study only three of the eight women discussed the issue of color directly. Interestingly, two of the three were the darkest skinned women in the study.

The issue of color discrimination first encroached on the research process when one of the women tried to determine its place in my life. Before my interview with Marcie, when we were engaged in the obligatory pre-interview chitchat, she inquired about my social background and my sorority affiliations. I knew immediately that this was part of "our community's code" for ascertaining where color stood in our lives. She identified my sorority as having a preference for light-skinned women. I assured her that this was not important to me and that I considered the idea of colorism as akin to having a slave mentality. To this I added that my sister, who is very dark, had been a member of this same sorority for 25 years and had actually pinned me. Nevertheless, Marcie let me know several times that my light skin stood between us as a source of tension by casually referring to the snobbishness of the "yella" girls "like me" at her exclusive all-women's school.

My coloring was again raised indirectly by Beth, who unexpectedly introduced colorism during the first few minutes of the interview and who commented on my wavy hair texture and length during our initial meeting. The difficulty of intraracial color discrimination was also advanced by her as an influence in her educational life. It was one of the first points Beth, a very dark-skinned woman, talked about in her school narrative. For Beth, her family's poverty was compounded by her dark skin color. When she was asked to describe her most unpleasant schooling experience, color was one of the primary topics she addressed: I guess, well the elementary years, I guess were not that pleasant because the fact that teachers didn't give people from single-parent homes that much attention and then it was based on the color factor. If you came, you know, if you were just lower socioeconomic ... the teachers didn't give me the time of day ... they had their favorites and their picks. I mean, even though my teachers were Black it was ... like a struggle of just trying to get ahead and ah, the ones that had less or wasn't bright enough [referring to skin color] just didn't get it.

Beth continued in her dissection of this problem stating that being "nappy headed" was another element in her life that complicated her acceptance. She referred to having "good hair" or naturally straight hair as "cutting the cake" or putting "icing on the cake." To her this was another stratum of colorism. Although she spoke of colorism as something that happened years ago, she later alluded to the difficulty her daughter presently faces in school because of her dark skin. Her tenseness of voice and the apparent pain in recounting the incidents were made evident by her tears. One of the most anxious moments in the research occurred when after describing the problems she had had as a

result of her dark skin, Beth insinuated that I would not have had similar problems because of my lighter skin color and "good hair." Unconsciously, and as a defense mechanism, I deliberately pinned my hair up to indicate its unimportance and discussed with Beth the pain that I had experienced around the issues of colorism. This discussion included people treating my sister differently because she was dark-skinned and the fact that I felt that skin color was relative to one's circumstances, explaining that my partner's fair-skinned freckled face and red-haired family members, who are also Black, considered me dark-skinned. This discussion seemed to relax her and opened both of us to a more intimate interview.

Color was not found in Sheila's or in Jean's stories. Neither of these light-skinned women spoke about this type of discrimination. It can be assumed that they were privileged by their skin color regardless of their feelings on the subject or a less likely premise could be that they were oblivious to its existence in society. Jean, however, did say that she was the queen of her Historically Black College or University (HBCU) in the early 1960s, and this notion of light-skinned Black women as the designated beauties of the race is supported by Russell, Wilson, and Hall (1992) who said that skin color was indeed a major component in Black beauty standards and a determining factor in the success of students at traditionally Black schools.

The only remaining respondents, Cathy and Lynda, did not address color as an obstruction in their lives. However, upon further analysis it is seen that they mentioned class as a definite determinant of their early and current life stations. According to researchers (Frazier, 1962; Russell, Wilson, & Hall, 1992), color is a confounding variable in determining the status of some Black families. They contend that because of severe colorism during Reconstruction and in the first half of the twentieth century a legacy of poverty continues to exist among dark-skinned families.[7] Colorism adds a layer to the multiple oppressions the women mediated in their school lives and personal lives

The participants and I spoke awkwardly about color. Even though I shared a lighter skin color with several of the participants, it was these participants who did not broach the subject of color and because the nature of the research was to solicit their schooling narratives, the topic was not introduced. Yet even in those interviews where it was not addressed, in analysis there appeared logical reasons for its absence. Privilege is a seductive entity. Those who cannot trade on the privilege of skin color are aware of it. Those who can trade on skin color are not aware of its inherent advantage. The participants who did raise the issue of colorism did so in an effort to determine its importance in my life. Although I tried to set forth that I saw colorism as a destructive factor in the history and present-day conditions of the Black race, colorism brought a tenuousness and a level of uneasiness to the interview setting. No matter how much I proclaimed to the women who raised the issue that I abhorred the discrimination, I could not in good conscience state that I had never benefited from it. Color remained a sometimes conspicuous and at other times inconspicuous barrier to conversational dialogues.

Unlike class, which is a known entity, colorism is a complicating consideration in the interviewing process. It can never be assumed to be present in the process simply because a participant is a person of color or because there are skin-color differences between the researcher and participants. Not all Black persons make or are affected by skin-color distinctions. But color as a societal force, complete with an internal hierarchial ranking system of preferences, is a part of the school narratives of the women in this study.

Certainly, color is dwindling in its importance in the Black community and it has always been individualistic in the nature of its importance to families, geographic regions, organizations, and people. Yet its presence as a potential obstacle in the person-to-person communication process between a Black researcher and a Black participant was an undeniable factor of the research project under discussion. If class was a wall in the research study to be peered across or broken down, then issues of color were most certainly occasional landmines in our mutual process of discovery.

Summary

The interviewing phase of qualitative research is dynamic and ever changing. No two situations or circumstances are ever alike. The most seemingly calm interview can prove unpredictable when the mixture of personalities and circumstances stir, energize, comprise or impact the setting in some manner. Although there are power issues that a researcher must remain cognizant of, such as balance of dialogue, research agendas, and societal hierarchies (Anderson & Jack, 1991; Cotterrill, 1992), basically each interview is a special unit of work unto itself. This does not change when women interview women, when Blacks interview Blacks, or when Black women interview Black women.

There are White researchers and male researchers who can interview Black women across the lines of race and gender with proficiency in spite of any differences that would separate or distinguish them from their interviewees. This study does not set forth that effective interviews cannot take place between a researcher and participant who are from vastly different backgrounds. However, commensurate with the intensity of the scenario when Black women talk within racial and gender boundaries are benefits. When there are fewer margins to mitigate, the research setting can take on an electrifying and intimate aspect. It was evident with the women in this study as race and gender stories were traded. During our talks, there were several times when different layers of the conversations were apparent. There were silent understandings, culture-bound phrases that did not need interpretation, and nonverbalized answers conveyed with culture-specific hand gestures and facial expressions laced throughout the dialogue. At times these shared issues of race and gender were connections between the women. However, periodically the mutuality of such understandings necessitated stepping back and asking for clarification because the dialogue was not to remain between the researcher and the interviewee.

There are no automatic guarantees that an interviewer will be accepted and that building trust will be easy. Interviewing remains work, even and sometimes especially when Black women interview other Black women. The differences of class and color, whether perceived or real, initially served as shackles on this explorative process of interviewing. Therefore, it was vital to the procedure that I was fully aware of the added forces of talking within my racial and gender group. But there were many more times when the experience of a Black woman interviewing a Black woman was advantageous. At these times the link of shared backgrounds and similar pains moved the women on both sides of the tape recorder to greater intimacy and trust and occasionally to tears.

Notes

1. Cotterill speaks to the psychological pitfalls when the relationship becomes too intimate. Minister considers the social systems involved when women interview other women and offers techniques to employ. Ribbens includes a discussion on the appropriateness of reciprocity in the interview process.

2. An unusual feature of the study is the researcher's role as participant. The work grew out of the researcher's position as a reentry woman and her interest in how other similarly situated women handled their academic sojourns. As a participant, the researcher was interviewed by her committee methodologist prior to going into the field. Both women studied and coded the data from this initial interview.

3. The theoretical framework of the study, Black feminist thought, as set forth by Patricia Hill Collins, requires that the research interviews be closer in format to a dialogue.

4. An analysis of the Black woman as superwoman is presented in The habit of surviving: Black women's strategies for life (Scott, 1991). The discussion explores the myth as a trap for Black women in terms of its inherent unfairness and built-in expectations.

5. For some Blacks, skin-color discrimination or colorism is disowned as an enigma that no longer afflicts the race. However, it has been described in The color complex (Russell, Wilson, & Hall, 1992) as an embarrassing dilemma that is present and actively a part of Black life in America.

6. The basis for this preference is the belief that a person who approximates a White person in skin color, facial features, and hair texture is superior in attractiveness, intellect, morals, and abilities. The corollary assumption is that Blacks with darker skin, broad facial features, and curly hair are inferior in the same areas, especially as regards intelligence and morals.

7. The correlation between poverty and dark-skinned Blacks is so endemic to the Black race that it is often overlooked in analysis. Therefore, it is quite possible that Cathy and Lynda, who are both from disadvantaged families, do not recognize dark skin as a possible complicating ingredient in their family histories.

---

Beverly Johnson. "Teacher-As-Researcher." ERIC Digest ED 355205. March, 1993.

 The concept of teacher-as-researcher is included in recent literature on educational reform, which encourages teachers to be collaborators in revising curriculum, improving their work environment, professionalizing teaching, and developing policy. Teacher research has its roots in action research.

 WHAT IS ACTION RESEARCH?

Action research is deliberate, solution-oriented investigation that is group or personally owned and conducted. It is characterized by spiraling cycles of problem identification, systematic data collection, reflection, analysis, data-driven action taken, and, finally, problem redefinition. The linking of the terms "action" and "research" highlights the essential features of this method: trying out ideas in practice as a means of increasing knowledge about and/or improving curriculum, teaching, and learning (Kemmis & McTaggart, 1982).

While the concept of action research can be traced back to the early works of John Dewey in the 1920s and Kurt Lewin in the 1940s, it is Stephen Corey and others at Teachers College of Columbia University who introduced the term action research to the educational community in 1949. Corey (1953) defined action research as the process through which practitioners study their own practice to solve their personal practical problems.

Very often action research is a collaborative activity where practitioners work together to help one another design and carry out investigations in their classrooms. Teacher action research is, according to John Elliott, "concerned with the everyday practical problems experienced by teachers, rather than the 'theoretical problems' defined by pure researchers within a discipline of knowledge" (Elliott, cited in Nixon, 1987). Research is designed, conducted, and implemented by the teachers themselves to improve teaching in their own classrooms, sometimes becoming a staff development project in which teachers establish expertise in curriculum development and reflective teaching.

The prevailing focus of teacher research is to expand the teacher's role as inquirer about teaching and learning through systematic classroom research (Copper, 1990). The approach is naturalistic, using participant-observation techniques of ethnographic research, is generally collaborative, and includes characteristics of case study methodology (Belanger, 1992).

The research study team provides support and a forum for sharing questions, concerns, and results. Teachers advise each other and comment on the progress of individual efforts. Engaging in collaborative action research helps eliminate the isolation that has long characterized teaching, as it promotes professional dialogue and thus, creates a more professional culture in schools.

WHAT IS THE PURPOSE OF TEACHER ACTION RESEARCH?

Action research has been employed for various purposes: for school-based curriculum development, as a professional development strategy, in preservice and graduate courses in education, and in systems planning and policy development. Some writers (i.e., Holly, 1990; Jacullo-Noto, 1992; Lieberman, 1988; Oja & Smulyan, 1989; Sagor, 1992) advocate an action research approach for school restructuring. Action research can be used as an evaluative tool, which can assist in self-evaluation whether the "self" be an individual or an institution.

WHY IS TEACHER RESEARCH IMPORTANT?

The current school restructuring movement has site-based, shared decision-making at its core. With the newly acquired autonomy, comes new responsibilities. Teachers, local schools, and school districts are accountable to all stakeholders for the policies, programs, and practices they implement. It is not enough for teachers merely to make decisions; they will be called upon to make informed decisions, decisions which are data driven. Therefore, it is necessary for teachers to be much more deliberate in documenting and evaluating their efforts. Action research is one means to that end. It is very likely the emergence of site-based decisionmaking has precipitated the resurgence of action research; the two seem to be complementary. Action research assists practitioners and other stakeholders in identifying the needs, assessing the development processes, and evaluating the outcomes of the changes they define, design, and implement. The self-evaluation aspect of action research (by educators and/or students) is congruent with the philosophies contained in the Total Quality Education and Outcomes Based Education movements currently being advanced by numerous states and districts throughout the nation.

WHAT ARE THE EFFECTS OF ACTION RESEARCH?

There is a growing body of evidence of the positive personal and professional effects that engaging in action research has on the practitioner (Goswami & Stillman, 1987; Lieberman, 1988). Action research provides teachers with the opportunity to gain knowledge and skill in research methods and applications and to become more aware of the options and possibilities for change. Teachers participating in action research become more critical and reflective about their own practice (Oja & Pine, 1989; Street, 1986). Teachers engaging in action research attend more carefully to their methods, their perceptions and understandings, and their whole approach to the teaching process.

Lawrence Stenhouse once said, "It is teachers who, in the end, will change the world of the school by understanding it" (cited in Rudduck, 1988). As teachers engage in action research they are increasing their understanding of the schooling process. What they are learning will have great impact on what happens in classrooms, schools, and districts in the future. The future directions of staff development programs, teacher preparation curricula, as well as school improvement initiatives, will be impacted by the things teachers learn through the critical inquiry and rigorous examination of their own practice and their school programs that action research requires.

Teachers' action research questions emerge from areas they consider problematic, from discrepancies between what is intended and what actually occurs. As Cochran-Smith and Lytle (1990) suggest, the unique feature of teachers' questions is that they emanate solely neither from theory nor from practice, but from "critical reflection on the intersection of the two" (p. 6). Teacher research will force the re-evaluation of current theories and will significantly influence what is known about teaching, learning, and schooling.

It has been said, "Teachers often leave a mark on their students, but they seldom leave a mark on their profession" (Wolfe, 1989). Through the process and products of action research teachers will do both.

Beverly Johnson. "Teacher-As-Reserach"

ERIC Digest: ED355205

Publication Date: 1993-03-00

---

Learning to Listen through Home Visits with Somali, Mien, Cambodian, Vietnamese and Latino Families

by Betty J. Cobbs and Margery B. Ginsberg

This article describes a summer learning experience that helped educational leaders listen to and learn from underrepresented voices. It provides a mosaic of insights contributed by 24 doctoral students from the University of Washington's Leadership for Learning (L4L) superintendent program. In July of 2005, these students, most of whom were currently principals, visited the homes of Somali, Mien, Cambodian, Vietnamese, and Latino families living in a Seattle-area school community. Although several of the students had visited low-income homes in their own districts, this was the first time they had done so with the primary purpose of listening to under-represented voices on matters of district policy.

The leadership students sought to understand how members of linguistic minority communities respond to policy decisions such a school closure. In addition to learning about perspectives on policy, home visits also helped students challenge assumptions regarding equitable communication and cultural awareness. The process revealed a structure for encouraging high-level decision-makers and administrators to meet with diverse families in their homes—one that we hope others will also consider using.

Approach

As part of a one-week collaborative action research project, pairs of doctoral students along with an interpreter visited two homes of families in the same language group. While one student carried out an interview of the family, the other wrote field notes. In the second home, they reversed roles.

The questions posed by the leadership students focused primarily on the topic of school closure. This was a timely issue because the school district was proposing the closure of several neighborhood elementary schools the following year. Questions in the semi-structured interview included the following:

What have you heard about school closings?

How do you get information about this and other school-related issues?

How do you imagine that sending your child to another school would influence your everyday life?

What questions would you hope to be able to ask the school district regarding school closure?

What community resources assist your children with school success?

In addition to the topic of school closure, what are your concerns as a parent who speaks a language that school leaders may not understand?

Is there anything else you would like to express that you may not have had an opportunity to communicate?

The Leadership students had the support of interpreters from the Seattle School District who speak languages from the five largest non-English linguistic communities. As recent immigrants themselves, the interpreters were able to provide a context for students to begin to understand the complicated histories and experiences with which families struggle. The interpreters went with students to visit Somali, Mien, Cambodian, Spanish, and Vietnamese homes. Each visit totaled two hours, including one-half hour on each end of the visit for informal conversation between the students and the interpreter.

In some ways, scheduling the visits was the most revealing part of the experience. It reflected the complicated schedules of family members and interpreters, many of whom worked two or three jobs. The interpreters were fully committed to connecting the Leadership students to families, despite existing demands. Through phone conversations complicated by the co-mingling of dialects and limited computer access, they helped identify and coordinate home visit times.

With the experience spanning five days, the first day was designated for preparation. Students reviewed literature on the topic of school closure (Rowley, 1984), read Funds of Knowledge (Moll, 1992), and practiced taking field notes (Glesne, 1999). They interviewed the interpreters from the linguistic community they would be visiting. Through role play and simulation, they practiced the interview experience from start to finish, serving as a guest and interviewer in the home of an under-represented family. The role play was intended to provide a context, though simulated, within which students could consider the influence of their power and the weight of their responsibility to the families they visited. Then, and only then, the visits began.

When the students were not visiting homes, they were on-campus preparing their final projects -- portfolios that included a review of literature, a series of ethnographic field notes, a field note analysis, a "Where I'm From" poem written in the voice of a child from one of the families (Christensen, 1997), and personal reflections. In language-specific discussion groups, the L4L students exchanged knowledge and insights.

Conversations with students and analysis of documents suggest that the experience influenced the students in several ways. They noted consistently that home visits provided a context for a) learning within an authentic, problem-based setting, b) deep listening to personal stories, c) making decisions about when and how to deviate from an interview schedule, d) becoming competent as new researchers, e) probing the limits of cultural awareness, and f) challenging personal assumptions about the lives of underserved linguistically diverse families.

The home visits also provided an opportunity to integrate concepts from various Leadership for Learning modules that occur throughout the three-year program. These modules include moral and historical foundations of education, organizational development and change, linguistic diversity, closing the achievement gap, special education, school law, and inquiry. Throughout the week, an overriding question students were asked to consider was: What is the significance of any of this to you as a future systems-level leader?

Shared Learning Themes

Cross-group ethnographic analyses of the students' personal reflections suggest eight primary learning themes. Each quotation below represents a theme that grew from the common student experiences across all five language groups, though there were other themes that applied only to a certain language group. The latter are not included in this discussion.

Theme One: Interpreters, as mediators of language and culture, are essential.

I heard over and over again during the interview process how important it is for families to communicate with people at the school who speak their language. The interpreters were a gold-mine of information—about culture, the needs of particular children and their families, how to navigate the public school system and American life.

Theme Two:Home visits can help educators become aware of how much we don't know, even when we think we do, about the families we serve.

I have had a false satisfaction that I was communicating with my parents when I was able to get school communiqués translated into different languages. But I was not communicating with them, I was communicating to them—because their own language tradition was primarily oral.

Theme Three: Written communication, even when it is translated, can fail to reach a large number of people, especially in linguistic groups steeped in rich oral tradition.

Rising above the language barriers and developing cultural understandings are necessary steps. But, even more basic than that, is taking the time to create processes that foster this communication. There is tremendous tenacity among linguistically diverse parents to seek translation from their children, relatives, neighbors and community resources regarding school matters.

Theme Four: Families are more than willing to welcome and interact with educators in their homes.

There was a sense of honor in having a "school official" visiting their home. There was a very welcoming attitude the minute you entered the room and there was also a sincere invitation to return to their home. I came away feeling humbled and in many ways grounded.

Theme Five: There is commitment and passion among families who are new to the United States.

To sit in a home of yet another family in this neighborhood whose aspirations must have been so similar to my non-English speaking grandparents was profound. A subtle kinship existed in my mind as I sat and listened to a mother's concerns for her children's schooling.

Theme Six: There seems to be a willingness to accept that school officials will make decisions that are in all children's bests interests.

For the most part, the families we visited seemed to have a lot of trust in their schools. When people are struggling to make ends meet and do not have information regarding schools, it would seem to me that this creates a special responsibility for those in positions of power (in this case, school officials). We have a moral obligation to advocate for families.

Theme Seven: Educational leaders play a critical role in ensuring that participation in policy-making involves everyone.

The key " take away" for me is that the people with whom we spoke really don't have a voice or communication path in our schools. Now that I know this, I need to ensure that I walk the talk of true partnerships between home and school. It is key to the success of children.

Theme Eight: Listening to stories is a challenge for systems-level leaders.

As we get further away from the classroom we tend to lose the personal stories that make up all of our work. Superintendent programs tend to focus on systems, and they tend to forget the stories that make up the system. It is important to get to know our students and families on an individual basis, and provide avenues in our schools and districts where families can become well known and have their voices heard.

Conclusion

Although the Seattle School Board decided to abandon the immediate plan for school closures, the experience of listening to families in their homes provided a powerful reminder about the importance and the luxury of communication. There are stories to be told, there are assumptions to examine, and there is always a responsibility to ask, "Whose interests are being served?" (Sirotnik, 1987). Try as we may to dignify success in schooling with test scores and bar charts, it is in relationships that educators become significant (Ginsberg & Wlodkowski, 2000).

Bibliography

Christensen, L. (1997). "Writing, and Rising Up: Teaching About Social Justice and the Power of the Written Word." Milwaukee, WI: Rethinking Schools, 12(2), pp. 22-23.

Ginsberg, M. B. & Wlodkowski, R.J. (2000). Creating highly motivating classrooms for all students. Jossey-Bass, San Francisco.

Glesne, C. (1998). Becoming Qualitative Researchers: An Introduction (2nd ed.). New York: Longman.

Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992)."Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms." Theory Into Practice, XXXI (2), pp. 132-141.

Rowley, S. R. (1984). "School Closure in Seattle: A Case Study of Educational Decision Making" (Doctoral dissertation, Stanford University, 1984).

Sirotnik, K. A. (1987). "Evaluation in the ecology of schooling: The process of school renewal." In J. I. Goodlad (Ed.), The ecology of school renewal: Eighty-sixth yearbook of the National Society of Education.

New Horizons For Learning January 2006

http://www.newhorizons.org

This article identifies and examines the ethical issues surrounding teacher research, especially when the participants of the research are the teachers' own students. I first explore the movement to increase the relevance and applicability of research on and for teachers, and then address ethical issues in teaching and in research, especially as they stem from federal regulation requiring the protection of human subjects. The article then turns to the specific issues that arise in teacher research. Dual-role conflicts are described, as are the difficulties of assuring unfettered informed consent. The article relates these problems to the difficulties of deciding what is research and what is normal educational practice in the classroom setting, especially when qualitative research methodologies are used. Suggestions as to how the potential conflicts and ethical problems can be addressed are provided, but teacher researchers are cautioned that work with their own students raises particularly thorny issues.

 This article identifies and examines the ethical issues surrounding teacher research, especially when the participants of the research are the teachers' own students. I first explore the movement to increase the relevance and applicability of research on and for teachers, and then address ethical issues in teaching and in research, especially as they stem from federal regulation requiring the protection of human subjects. The article then turns to the specific issues that arise in teacher research. Dual-role conflicts are described, as are the difficulties of assuring unfettered informed consent. The article relates these problems to the difficulties of deciding what is research and what is normal educational practice in the classroom setting, especially when qualitative research methodologies are used. Suggestions as to how the potential conflicts and ethical problems can be addressed are provided, but teacher researchers are cautioned that work with their own students raises particularly thorny issues.

Current efforts aimed at improving education are increasingly emphasizing the need to further develop and codify the knowledge base of teaching. Additionally, reformers focusing on teachers have emphasized the relative isolation of teachers from each other and the need to develop new venues for teachers to interact around the problems of practice they face (Shulman, 1990). Such interaction is seen as essential for another goal of reform: professionalization of teaching (Holmes Group, 1990). One element of a profession frequently identified in this literature is the possession of a body of knowledge unique to the profession, and based on both basic and clinical research specific to the tasks of the profession. Thus, a movement to identify forms of teacher scholarship and writing (and talking) as research and to encourage teachers to see research as a part of their roles as teachers has gained momentum (Cochran-Smith & Lytle, 1992, 1993; Curriculum Committee of the Holmes Group, 1991; Darling- Hammond, 1993; Duckworth, 1986; Evans, 1991; McElroy, 1990; Patterson, Santa, Short, & Smith, 1993).

The link between the professionalization of teaching and increased research on the problems of practice is clear. Professionalized occupations claim to be experts in their practice, holding knowledge and practice skills beyond the capacity of laypersons. Thus research or scholarship of some kind is necessary to demarcate their expertise from that of the general population. The scientific base of medicine or the esoteric knowledge of a lawyer or of a priest, rabbi, or minister (although these are not science-based) is beyond that of most patients, clients, or believers, and supports the special, professional status of these occupations in the community. The need for involvement by teachers in research is not immediately self-evident from the uses other professions have made of research. Certainly not all doctors, rabbis, or engineers engage in research or otherwise make contributions to the knowledge base of their professions. Someone needs to do that work, but the general practitioners of the professions usually are not the ones.

The movement toward involving teachers in research derives from a critique of university-based research and researchers who have often defined educational problems in isolation from real classrooms and real students and teachers (Cole & Knowles, 1993). In the field of educational sociology, for example, the relations between educators and sociologists have been referred to as "uncomfortable" (Hansen, 1967). Research contributing to the discipline of educational psychology has sometimes been seen as more important than contributions to the classroom work of teachers (Saphier, 1982). Thus, the critique of teacher education and of schools of education has led to proposals for closer collaboration between schools of education and schools and between university faculty and classroom teachers. Such collaboration, it is argued, will produce knowledge more sensitive to the realities of teachers' tasks and the learning environment of classrooms. "Teachers have begun to assume their rightful position at the head and heart of inquiry into classroom practice" (Cole & Knowles, 1993, p. 477).

The thrust of this line of argument is different from the one emerging from a motivation to professionalize teaching. In some ways, this line of argument is almost antiprofessional and almost always antihierarchical, calling for closer relations between researcher and participant: "The university researcher may not always shape the nature and direction of the work. Instead, choices, change, and understandings are shared and shaped by everyone involved in the inquiry process" (Clark & Moss, 1996, p. 521).

Engagement in research, however, may place classroom teachers in roles they had not anticipated, and that may conflict with their roles as teachers (Baumann, 1966; Wilson, 1995; Wong, 1995a, 1995b). In particular, the potential for ethical problems for teachers resulting from dual relationships with their own students—as teachers of students and as researchers with subjects—is particularly severe.1

E. David Wong (1995a) describes a situation where the conflicts of interest are readily evident. A researcher concerned with how students develop understanding of natural phenomena, Wong taught a high school science class to better experience learning in a natural setting. While he was discussing an experiment with the class, one student's answers became the focus of his attention. He seized on what seemed to him a particularly illuminating moment for his research, and his concern as a researcher was to allow the student the time to develop her thoughts while holding the rest of the class at bay, preventing them from jumping in with their own ideas. As he pursued the data the one student was providing him, the rest of the class became increasingly distracted, frustrated in their inability to participate and not knowing or understanding why the teacher was stopping their involvement. Reflecting on the experience, Wong writes, "My actions as a researcher compromised my role as a teacher" (Wong, 1995a, p. 26). The conflict Wong describes here is a practical one, between the demands of teaching and those of his research, with ethical implications. Baumann (1996), too, lists a number of occasions when time constraints limited his ability to satisfy his roles as teacher and as researcher, and how these conflicts affected his performance in both roles.

Dual-role conflicts do not arise when research participants are not otherwise related to the researcher, but are present when a teacher uses his or her own students in research. Research, as opposed to practice, is undertaken primarily for the purpose of discovery. Thus, when one does research, one takes on a role that is different from that when one practices. For research to generate knowledge not otherwise known, new variables, behaviors, activities, and so forth, are introduced and examined; new questions lead to new ways of approaching or defining data. Something other than normal practice happens when research takes place; otherwise, it would not be called research. Certainly, in the case of schools, the normal activity is not research, but teaching and learning. The difficulty of determining what distinguishes practice from research will be raised below. For now, it is important to acknowledge that teacher research implies activities other than normal practice and that the dual roles of teacher of students and researcher with research participants may lead to conflicts (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979; Code of Federal Regulations, 45 CFR 46, 1983).

Dual-role conflicts will be become more common, I anticipate, as the rewards of doing research increase and become more prevalent in the professional lives of teachers. It must be recognized that there are rewards for doing research. The visibility and opportunities for further professional growth and development that often flow from research and publication offer real incentives, especially in teaching, where few alternatives to classroom work are available for most teachers. Those who become involved with research may develop an "interestedness" in research that can compete with the obligation to keep the "interests" of their students paramount.

ETHICS OF TEACHING AND OF RESEARCH

Teachers' primary obligations are to their students,2 while researchers have obligations to the field to which they seek to make a contribution. As researchers, teachers expand the audiences for their work to include not only students, but peers, administrators, and colleagues outside their immediate place of employment. The preparation of papers, books, workshops, and so forth, is an essential part of the researcher's task. Employing research techniques acceptable to the field is necessary, and requires a commitment to norms and canons of research in the field to which one hopes to contribute. The incentives to do research and to disseminate one's findings are significant, especially for teachers seeking professional involvement outside the classroom.

Virtually all professional occupations have formalized statements of ethics that are applied to the conduct of members of the profession. These codes serve two purposes. First, they help mark off the "profession" from other occupations. Thus, they place the profession in a category of work with special obligations and special status, stemming from their powers (the knowledge and practices only they can exercise: e.g., writing prescriptions) and deserving of higher status than other occupations. Second, codes of ethics are attempts to make explicit that the practitioner of the profession is committed to the welfare of the profession and of the client over all other considerations. Professional actions are to be taken not for the self-interest of the practitioner, but for the benefit of the client. This commitment, not found in such occupations as used-car sales, is intended to provide a foundation of trust between the practitioner and the client. Such trust is seen as essential for the professional's work to be successful (Abbott, 1983).

At the same time, codes of ethics also specify in general terms the obligations of practitioners to clients. The rationale for general specification—a s opposed to specific—is the allowance for professional discretion: professional authority and autonomy exist because of the superior knowledge of member professionals about how to address problematic circumstances facing their clients. While standards of "best-practice" may exist, professions are loath to codify these prescriptions as ethical obligations of practitioners toward their clients. Core values sustaining individual "professional judgment" are at stake, and have been strongly defended. Clear abuse of clients or patients, however, is always prohibited and subject to the harshest discipline (Abbott, 1983).

For teachers, especially teachers in elementary and secondary schools, where children are compelled by law to attend, the moral dimension, however codified, is an essential aspect of the work. Soder (1990) concludes that "it is precisely because children are compelled and children are defenseless and have low status that teaching has moral obligations and thus moral praiseworthiness" (p. 74). Children are deserving of special protection because of their age and because they do not have free choice to attend school.

Teacher codes of ethics as developed by various teacher organizations are generally consistent with the patterns Abbott (1983) described and share Soder's (1990) views. While concern for the ethical dimensions of education and of teaching is on the rise (Cuban, 1992; Goodlad, Soder, & Sirotnik, 1990; Journal of Teacher Education, 1991; Strike & Soltis, 1985), thus far this focus has not included the realms of research. The National Education Association's (1989-1990) Code of Ethics contains two principles: Commitment to the Student and Commitment to the Profession, each with eight provisions. Involvement in research is not mentioned, but teachers are admonished not to "use professional relationships with students for private advantage" (p. 317). Also, they "shall not disclose information about students in the course of professional service" (p. 318). These two provisions clearly relate to the potential for dual-role conflicts if research is undertaken by a teacher.

The recently issued "Code of Ethical Conduct and Statement of Commitment" of the National Association for the Education of Young Children asserts, among other principles, the association's commitment "to appreciate the special vulnerability of children . . . to create and maintain safe and healthy settings that foster children's social, emotional, intellectual, and physical development and that respect their dignity and their contributions" (in Feeney & Kipnis, 1989, p. 26). The code also contains statements of principles regarding research and confidentiality of student records. About research, the principle is stated thus:

We shall not permit or participate in research that could in any way hinder the education or development of the children in our programs. Families shall be fully informed of any proposed research and shall have the opportunity to give or withhold consent. (p. 27)

This provision requires that parents be notified before any research takes place, that they be informed about the purposes of the research and what it will entail for their children, and that they be offered the opportunity to opt in or out of the research.

The recently issued "Ethical Standards of the American Educational Research Association" (1992) has six "guiding standards," each followed by a "Preamble" and specific "Standards." The second guiding standard includes standards concerning research populations. The preamble includes the following language:

It is of paramount importance that educational researchers respect the rights, privacy, dignity, and sensitivities of their research populations. . . . Educational researchers should be especially careful in working with children and other vulnerable populations. These standards are intended to reinforce and strengthen already existing standards enforced by institutional review boards and other professional associations. (p. 23)

Of particular interest for the concerns of this article, standard 5 asserts that "educational researchers should exercise caution to ensure that there is no exploitation for personal gain of research populations or of institutional settings of research. Educational researchers should not use their influence over subordinates, students, or others to compel them to participate in research" (p. 23). How teachers can create such conditions in their classrooms and conduct research with their own students is a crucial question to which we shall return.

Developed formally during the 1970s, the regulations of the Department of Health and Human Services for the protection of human subjects now specify the requirements researchers using human subjects must comply with in carrying out federally funded research. These rules and the procedures they specify were enacted to prevent abuse of subjects by medical and psychological researchers who had often put their research objectives before concerns for their subjects.3 While research employing dangerous procedures with high risk or deception, especially when carried out with dependent or vulnerable groups as research subjects (e.g., prisoners, children, the seriously ill), has caused the greatest concern, the rules cover all types of research with human subjects.4 Because these rules specify the way research funded with federal funds may be conducted, they have come to be applied to all research with human subjects at many institutions, such as colleges and universities, and by professional associations such as the American Psychological Association. The Code of Ethics of the American Educational Research Association (1992) makes explicit reference to institutional review boards (IRBs), entities created at institutions where research takes place in compliance with the relevant federal regulations.

In essence, these rules begin by requiring an assessment of all risks the research may generate, and an assessment of any risks against benefits that may be produced.5 Who and or what will benefit from the research should be explicit. The researcher must obtain the informed consent of research subjects to be participants in a way that is without coercion and that treats the information they provide confidentially.6 The procedures usually call for the establishment of committees, often referred to as institutional review boards, composed of researchers and public representatives, to review research proposals and to certify that each proposal conforms to the regulations (Sherman & Van Vleet, 1991).

There has been some controversy over specific aspects of the regulations and the review process (Fluehr-Lobban, 1994; Thompson et al., 1981; Wax & Cassell, 1979). Social science researchers and associations have been the most concerned, and have identified several ways in which these regulations seem to pose a threat to their conduct of research. For example, researchers studying illegal behavior have found gaining prior, informed consent difficult at best (Punch, 1986). Others think the rules are overly bureaucratic and, for fieldwork, are held to be inappropriate regulations that may do more harm than good (Cassell, 1979). More recently, Clark and Moss (1996) appear to reject the rules regarding informed consent, arguing that they were formulated for "fixed procedures," while fieldwork methods in collaborative research must be flexible, and, quoting Wax (1980), "sequential and conditional so that consent is a continual process, dependent upon mutual learning and development" (p. 275).7

It is worth noting that this view is not universal among anthropologists and other fieldworkers. Fluehr-Lobban (1994) titled her lead article in the journal Human Organization (the journal of the Society of Applied Anthropology) "Informed Consent in Anthropological Research: We Are Not Exempt." Addressing concerns such as those expressed by Clark and Moss (1996) and Wax (1980), she concludes that in anthropology there are no important limitations imposed by informed-consent regulations. She argues that another source of resistance to the regulations stems from paternalism. "In social-cultural anthropology or applied research, under conditions of colonialism or other forms of subjugation, paternalism had an historical context, which was always an unequal power encounter between the anthropologist and the subject. Colonialism has ended and the formal rationale for paternalism has disappeared" (Fluehr-Lobban, 1994, p. 8). Yet other forms of unequal power remain, such as those between teachers and their students.8

Interestingly, the ethical code of the National Association for the Education of Young Children (in Feeney & Kipnis, 1989) virtually endorses paternalism as a professional obligation, given the "special vulnerability of children" (p. 26). This stance is rejected in anthropology by Fluehr-Lobban (1994) as a holdover from the "My Tribe" syndrome. Nothing in her article, however, related to children, nor to situations where the researcher has other obligations to the participants, as in the dual-role conflicts of teacher-researchers. The issue here centers on the dual roles of teacher-researchers. The exercise of the teacher role may require protecting children from some knowledge or information that could interfere with their judgment, were they fully competent adults. Such protection is defensible given the teacher's ethical obligations to children. For a researcher, keeping such information from a participant violates the ability to make an informed choice, and has lost its justification, according to Fluehr-Lobban.

The federal regulations have their origin in a statement of research ethics called The Belmont Report (National Commission, 1979). Appointed to study the increasing ethical complexities of research, the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research sets out in its report the basic ethical premises the current regulations embody. Essentially these premises follow three ethical principles: respect for persons, beneficence, and justice. Each of these principles will be elaborated on and applied to the research setting.

Respect for persons asserts that individuals should be treated as autonomous persons, and that persons with diminished autonomy are entitled to protection. In the context of research, this principle alerts researchers to the need to inform potential participants of what the research will entail so that they can make an unfettered decision to participate. Moreover, any constraints surrounding the decision to participate, such as fear of the consequences of not participating, must to be removed so that the decision is, in fact, a freely made one. Again, the problem of classroom teachers' using their own students as research participants is evident.

Some people have limited autonomy, however, and special considerations are necessary under the principle of respect for persons. Children, sick people, those dependent on institutional services, and those otherwise incompetent are entitled to greater levels of protection from the consequences of decisions that may not be made freely.

The principle of beneficence directs researchers to do no harm and to maximize benefit for participants and for society at large. Any risks posed by the research must be carefully identified by the researcher and lessened as much as possible. These risks, and any benefits that may follow, must be clearly communicated to potential participants before they agree to participate.

The principal of justice requires that one think through the burdens and benefits of the research and assure that there is fairness in the distribution of each. Is the research using only participants who are easily available to the researcher because of their compromised position or docility, rather than for reasons directly related to the problem being studied? Is research that will benefit all being carried out only on the poor or on the homeless? Justice requires that the benefits and burdens of research be shared.

For classroom teachers doing research with their own students, these principles raise several thorny problems.

PROBLEMS WITH TEACHER RESEARCH

The potential for ethical problems is exacerbated as a result of the development of nontraditional research methodologies that seek to break down the distinction between researcher and practitioner and subject (such as described by Bogdan & Biklin, 1982; Cole & Knowles, 1993; Lincoln & Guba, 1985, 1989; Yin, 1989). These new styles of research, which challenge traditional conceptions of research, are gaining in popularity—case studies of teaching and the analysis of a researcher's own teaching among them (Brandt, 1992). Instead of the probabilistic results of process-product research, Shulman (1990), for example, argues for research designs that yield "stories" that detail the "wisdom of practice" (p. 15).

Researchers utilizing any research methodology need to be able to describe their research in terms that are accessible to potential subjects and their parents, in order to obtain informed consent.9 These newer designs, and the collaborative and joint teacher-researcher programs intent on producing open-ended participation of both researcher and subject, however, are frequently hard to define for laypersons, who are likely to have experimental or survey models of research in mind. Thus, being clear about the purposes and procedures of the research to subjects may not be particularly easy, and especially so for children.

The primary issue to be explored here, however, is the nature of the teacher-researcher's commitment when in a dual-role situation. A teacher normally charged with the education of a class is, or should be, committed to educational practices that serve the best interests of those students. Although teachers are employees of school districts, and thus must relate to administrators, the board of education, and parents, their professional clients are their students. Decisions about what happens in the classroom, influenced by state law and district and school policy and objectives, should be, nevertheless, determined by what is best for the student. Cuban (1992) describes this aspect of teaching as one that requires "making concrete choices among competing values for vulnerable others who lack the teacher's knowledge and skills, who are dependent upon the teacher for access to both, and who will be changed by what the teacher teaches, how it is taught, and who the teacher is" (p. 9). The teacher's own span of autonomous decision making may be relatively limited, but the exercise of what autonomy exists is justified by the criterion of ethical service to the student specified in the codes of conduct under which teachers practice.10

The problem raised by doing research with one's own students is that the criterion of service to the student is potentially jeopardized by needs determined by the research, as Wong (1995a) and Baumann (1996) have illustrated. When one anticipates that the outcomes of research will include preparing papers for professional meetings and publishing the findings, the research must meet the substantive and technical canons accepted by the discipline, or the paper will not be accepted for presentation or for publication. The teacher-researcher, then, may face a situation where a classroom action has two potential rationales: what is best for the student or the class or what is needed for the research. While one hopes that each of these criteria will result in the same decision, Wong and Baumann describe how hard this may be in real classroom settings. When the decisions are not the same, the teacher-researcher is in a dual-role conflict situation.

As teachers begin to take on research roles in their classrooms, moreover, another complexity may evolve. The focus of much teacher research has been on teaching itself and the line between the two roles is often blurred, both in fact and in the teacher-researcher's own perception. Thus, where the roles of teacher and of researcher may be quite distinct in other settings, and the obligations of each role to the student-participant clear, when teachers conduct research in their own classrooms, the lived experience is a blending of the two. This ambiguity may confound teachers' efforts to enact both roles responsibly, as Wong's (1995a) example illustrates. It can even make the potential for conflict more difficult to perceive.

One aspect of the federal regulations (Code of Federal Regulations, 45 CFR 46, 1983) can illustrate another difficulty involved in teacher research in the classroom—whether the teacher is at a school or a college. A central tenet of the regulations for the protection of human subjects, and one that follows from the principle of respect for persons, is that subjects must have the opportunity, free of coercion, to decide whether to participate in the research. This decision is to be "informed," made on the basis of information about the research, its purposes, and its risks and potential benefits. The usual way of affording potential participants this opportunity to choose is by providing a written description of the research objectives and what the research will entail to the prospective participant and asking the participant to sign a consent form. In the case of minors, a form also must be signed by a parent or guardian.11

In usual practice, where participants are drawn from the ranks of those served by an organization or agency (such as schools, hospitals, clinics, or social service agencies), the decision to participate or not must not alter the right to or quality of service the client would otherwise receive. In conformity with the principle of respect for persons, the decision to participate in the research must be made in a noncoercive atmosphere. In classrooms and in other places where researchers have other relationships (such as between teacher and student with those they are studying), however, such circumstances are difficult to arrange (Marshall, 1992; Thorne, 1980; Wax, 1980).

For students in a class in which the teacher-researcher intends to carry out a research project, the freedom to decide whether to participate is severely compromised. Clearly, if the teacher is doing the research, then he or she is very much involved with it. A student who says no or who seeks to drop out once the research has commenced risks the disappointment, or even the wrath, of the teacher. If the teacher wants all the students in the class to be involved, the pressures to participate may be very great. These may come not only from the teacher, principal, or superintendent, but also from fellow students.12

If students or their parents choose to opt out of the research being undertaken, the ethics of teaching assert that this decision must not have negative consequences for the children. Moreover, provisions should be made so that the children who do not participate are not left out of important classroom activities or subject to detrimental treatment by the teacher-researcher or other students.

One would hope that the decision to participate or not could itself be confidential, and not a matter of public knowledge among other members of the staff or fellow students. It is in just such considerations that dual-role conflicts lurk. Researchers generally seek to maximize participation in their research, whereas teachers are likely to maximize learning opportunities. The natural inclinations of the two roles are not the same.

If the only relationship the teacher has with his or her students is as teacher, and an outside researcher requests access to recruit and research students, the teacher's attitude toward the research would be likely to err on the side of protecting students from research participation, unless a clear benefit would be forthcoming to the students, to the school, or to instruction. When dual relationships are involved, teacher/student and researcher/subject, the potential for conflict between priorities is very real.

In addition, the relationship between a teacher in a school (which most students attend under penalty of law, as well as, perhaps, out of choice) and students is not symmetrical. As Cuban (1992) stressed, the teacher grades students, and the school influences the students' educational and occupational futures, not the other way around. Therefore, to envision a situation in which the student's consent could be obtained without the perception by the student of constraint, if not coercion, is at best difficult.

The ethical obligations teachers have as teachers in serving the needs of their students seem paramount. Thus, the potentially coercive nature of teachers as researchers when they request student participation in their own research creates problems that must be confronted.

Because of the conflict when the participants are in a relationship of dependency with the researcher, the Committee for the Protection of Human Participants in Research (1982) of the American Psychological Association recommends to its members "turning over the recruitment of participants and the conduct of the research to a third party not involved in the relationship" (p. 51). In other words, this professional body recommends that research not be done with clients who are dependent on other services of the researcher or his or her employer.13

WHAT IS NORMAL EDUCATIONAL PRACTICE AND WHAT IS RESEARCH?

Another way to conceive of the problem identified here is to address the question of what constitutes "research." For many teacher-researchers, the line between normal practice and research is a fine one (Wilson, 1995). Good teaching practice has always required close observation and experimentation. Moreover, many of the current efforts to encourage teachers to engage in research fall into the category of systematic professional introspection and reflection—of research about one's own teaching. The increased use of the term "inquiry" where research might have otherwise been used seems to reflect this view. Clark and Moss (1996) treat ethical and epistemological issues together, asserting that to do otherwise would "mark a shift away from research done ‘with' teachers and students back toward a method of treating these groups as the objects of study" (p. 522). Baumann (1996) even goes so far as to say that teacher research should be considered a new genre of research. Yet the difficulties of drawing lines between the activities of teaching and of doing research remain, as Baumann also notes.

The dividing line that is suggested here is provided by the answer to the following question: Would the activities undertaken for the research have been carried out anyway, even if no research outcomes (e.g., professional presentation or written research paper) were sought? Researching what one does anyway emphasizes the anyway—that is, the teaching, not the research. In this way, potential conflicts between teaching and research will be reduced. They probably cannot be eliminated.

The authors of The Belmont Report (National Commission, 1979) raised the issue of what constitutes practice and what constitutes research at the beginning of their text. Noting that the distinction between these two functions is blurred, they suggest that the general rule to be applied should be as follows: "If there is any element of research in an activity, that activity should undergo review for the protection of human subjects" (1979, p. 3).14

In the regulations, research is defined as "a systematic investigation to develop or contribute to generalizable knowledge" (Code of Federal Regulations, 45 CFR 46, 1983, p. 5). Whether the newer forms of teacher research and of action research by teachers would belong in this category is a question that should to be addressed. However, the dual-role conflict would remain in either case.

As noted earlier, many of the models of research now guiding teacher-researchers and their university collaborators are based on assumptions regarding the conduct of research that are different from traditional, and most medical, research—the model of research assumed by the authors of The Belmont Report (National Commission, 1979). Whether one refers to participant observation research, action research, feminist research, or post-positivist research, the paradigm of scientific hypothesis testing and experimental or quasi-experimental design is not universally assumed. Cole and Knowles (1993), for example, assert that "hypothesis testing and theory generation by rating, classifying, and correlating observational and verbal report data gathered under contrived (or at least controlled) conditions are no longer readily accepted ways of apprehending and representing teaching and classroom life" (p. 475). They go on to state:

Through the transition in the way educational research activities are viewed, research into teaching has become more "subjective," more intensive, more demanding of the participants and less controlled by researchers affiliated with academic and research institutions. In some cases, research has become more personally and professionally intrusive. . . . New forms of partnership research are based on fundamental assumptions about the importance of mutuality in purpose, interpretation, and reporting, and about the potency of multiple perspectives. (pp. 477-478)

To their credit, these authors note that this new research style needs to address several ethical issues, including assuring confidentiality of informants. Although they do not describe what the dilemmas were, Clark and Moss (1996) also say: "We have encountered dilemmas that have forced us to make ethical choices"(p. 525). How consent will be given by research participants is another issue.

These are, indeed, important questions if one is going to do research, no matter which style or paradigm one adopts. To them we need to add the question of how vulnerable, dependent populations, often unable to negotiate with school authorities and with little capacity to collaborate as equals, are going to be protected. How can we assume that mutuality of interest really exists when power differentials between students and their teachers are great and usually irreducible? Simply defining "subjects" as "participants," or calling what one does inquiry instead of research, does not change the fundamental differences in status. As Baumann (1996) puts it, "My point is that I do not believe that conflict in teacher research is methodologically dependent" (p. 32).

Moreover, if the new research styles are more intrusive, by definition the risks of participating increase; otherwise, "intrusive" has no meaning. The principle of beneficence requires researchers to make explicit the risks to potential participants and to assure as much as possible that no harm be done. Whatever the risks of harm might be from more intrusive research, they must be balanced against the benefits the research offers, either directly to the participants or to future teachers or students.

What remains important, in my mind, is that students come to school expecting to learn, to be taught, not to be subjects—or participants—of research. We compel them to attend and tax their parents and other citizens to support our educational activities on the assumption that we have some idea of what we are doing. Increasing that knowledge is essential, but that is not the primary reason schools exist. Increasing knowledge about health is not why hospitals exist. Rather they exist and are supported to provide the relevant professions a place to practice. In fact, the analogy between hospitals and schools is a flawed one. Learning in school is a normal part of one's life; being ill or otherwise needing the services of a hospital is not normal (except, perhaps at birth or death).

Returning to the question of what is practice and what is research helps to clarify some of the issues raised here. It points out the limits of the knowledge we use to practice and alerts us to be mindful when we engage in activities that are not "normal educational practice." Perhaps most importantly, we are alerted to the world as perceived by our students; do they know the difference between practice and research? When are they free to choose which to experience? Their very lives may not be at stake as if they were choosing to treat a cancer with experimental radiation treatment over surgery, but it is simply incorrect to assume that there are no consequences to their less-threatening choices, or that those choices are made freely.

Nothing said here is intended to deprecate the value of more research undertaken in real schools and in real classrooms. But we cannot have it both ways. With our own students, we must remain teachers—teachers who are alert and professional, who are constantly trying to perfect their practice and who experiment and reflect. Certainly all of that falls within "normal educational activities," especially when done within the context of the school's normal processes of decision making.15 In fact, it can be argued that much of what is referred to as teacher research is normal educational practice, or should be. At the same time, we need to be aware of limits to such practice, and to when and where lines between normal practice and research should be drawn.

WHAT SHOULD BE DONE?

Teachers who also wish to be researchers must think about the rights of their students and other participants of their research activities and become familiar with the rules and procedures for the protection of human subjects. Where research evolves as a frequent activity of teachers and other school-based professionals, those involved in the research should be members of institutional review boards. Currently, many schools refer requests to conduct research from outside researchers to their central board, where a procedure for clearance is often already established. However, while the central office is where researchers have been located, as teachers themselves become more involved in research, this centralized procedure should include representatives from the research site at the school level. This notion is in concert with increased teacher professionalism and site/school-based management systems. Moreover, examination of ethical issues surrounding classroom research is best done when those who know the context well—teachers interested in research—are involved. It should also be done in a public way (Bok, 1978).

These review boards may benefit from coordination with university IRBs linked with the school, such as in professional development schools. Shared responsibility for protection of human subjects is common between hospitals and clinics and universities now, and relations among schools and universities should be established.

Finally, we need more thought about "normal practice" and establishing canons of "clinical research." Reflective practice does necessitate record keeping and analysis, but schools do that anyway. Curricula must be evaluated, and innovative teaching practices assessed. Student records must be shared and maintained in a methodical way. All these activities are a part of normal educational practice. It is when teacher-researcher activities are not a part of the school's normal educational processes that the dual-role conflicts arise. And it is then that conflicts of interest between teacher and researcher roles may jeopardize the professional obligations of teachers to their students. The blending and blurring of teacher and researcher roles referred to above make the need to think through our research and its consequences critical. That teacher research may be based on new paradigms and utilize more informal methods does not reduce or eliminate the problems and conflicts that may arise. The "intrusive" nature of some of this research (Cole & Knowles, 1993) alerts us to the potential problems. Only teacher-researcher awareness of the possibility of dual-role conflict and of issues concerning the protection of human subjects in research can safeguard our students from the bind we may set for them as we conduct research in our own classrooms.

Much is to be gained from greater involvement of classroom teachers in research. Their continuing professional development can be enhanced; their enthusiasm and commitment to teaching can be stimulated by adding a new dimension to their work; their ability to participate in school-level decision making can be improved. All of these rewards should benefit their students as well. As teacher research expands, however, teachers need to keep alert to the ethical issues that may arise and in particular those surrounding dual-role conflicts.

This article is a revision of a paper read at the 1992 annual meeting of the American Educational Research Association in Atlanta, Georgia. The comments of commentators and other participants at those meetings were very helpful. In addition, I am grateful to Marti Dunne, Joseph B. Giacquinta, Ann Greenberg, Ivor Pritchard, Jerrold Ross, Betsy Useem, and several referees for suggestions for improving this article.

REFERENCES

Abbott, A. (1983). Professional ethics. American Journal of Sociology, 88, 855-885.

American Educational Research Association. (1992). Ethical standards of the American Educational Research Association. Educational Researcher, 21(7), 23-26.

Baumann, J. F. (1996). Conflict or compatibility in classroom inquiry? One teacher's struggle to balance teaching and research. Educational Researcher, 25(7), 29-36.

Bloom, L. R. (1997). Locked in uneasy sisterhood: Reflections on feminist methodology and research relations. Anthropology and Education Quarterly, 28(1): 111-122.

Bogdan, R., & Biklan, S. K. (1982). Qualitative research for education: An introduction to theory and methods. Boston: Allyn & Bacon.

Bok, S. (1978). Lying: Moral choice in public and private life. New York: Random House.

Brandt, A. M. (1978). Racism, research and the Tuskegee syphilis study (Hastings Center Report 8) Hastings-on-Hudson, NY: Institute of Society, Ethics, and Life Sciences.

Brandt, R. (1992). On research on teaching: A conversation with Lee Shulman. Educational Leadership, 49, 14-19.

Cassell, J. (1979). Regulating fieldwork: Of subjects, subjection, and intersubjectivity." In M. Wax & J. Cassell (Eds.), Federal regulations: Ethical issues and social research (pp. 129- 144). Boulder, CO: Westview.

Clark, C. T., & Moss, P. A. (1996). Researching with: Ethical and epistemological implications of doing collaborative, change-oriented research with teachers and students.

Cochran-Smith, M., & Lytle, S. L. (1992). Communities for teacher research: Fringe or forefront? American Journal of Education, 100, 298-324.

Cochran-Smith, M., & Lytle, S. L. (Eds.). (1993). Inside/outside: Teacher research and knowledge. New York: Teachers College Press.

Code of Federal Regulations, Title 45—Public Welfare—Part 46—Protection of Human Subjects (45 CFR 46). (1983). Revised. Washington, DC: Department of Health and Human Services, National Institutes of Health, Office for Protection from Research Risks.

Cole, A. L., & Knowles, J. G. (1993). Teacher development research: A focus on methods and issues. American Educational Research Journal, 30, 473-495.

Committee for the Protection of Human Participants in Research. (1982). Ethical principles in the conduct of research with human participants. Washington, DC: American Psychological Association.

Cuban, L. (1992). Managing dilemmas while building professional communities. Educational Researcher, 21, 4-11.

Curriculum Committee of the Holmes Group. (1991). Toward a community of learning: The preparation and continuing education of teachers. East Lansing, MI: The Holmes Group.

Darling-Hammond, L. (1993). Professional development schools: Schools for a developing profession. New York: Teachers College Press.

Diers, D. (1990). Editorial: on mischief . . . . Image: Journal of Nursing Scholarship, 7, 138.

Duckworth, E. (1986). Teaching as research. Harvard Educational Review, 56, 481-495.

Evans, C. (1991). Support for teachers studying their own work. Education Leadership 48, 11-13.

Feeney, S., & Kipnis, K. (1989). A new code of ethics for early childhood educators! Young Children 45, 24-29.

Fenstermacher, G. D. (1990). Some moral considerations on teaching as a profession. In J. I. Goodlad, R. Soder, & K. A. Sirotnik (Eds.), The moral dimensions of teaching (pp. 130-151). San Francisco: Jossey-Bass.

Fluehr-Lobban, C. (1994). Informed consent in anthropological research: We are not exempt. Human Organization, 53(1), 1-10.

Goodlad, J. I., Soder, R., & Sirotnik, K. A. (1990). The moral dimensions of teaching. San Francisco: Jossey-Bass.

Grinnell, F. (1990). Endings of clinical research protocols: Distinguishing therapy from research. IRB: A Review of Human Subjects Research, 12(4), 1-4.

Hansen, D. A. (1967). The uncomfortable relation of sociology and education. In D. A. Hansen & J. E. Gerstl (Eds.), On education: Sociological perspectives (pp. 3-35). New York: John Wiley.

Hauser, M. E. (1997). How do we really work? A response to "Locked in uneasy sisterhood": Reflections on feminist methodology and research relationships. Anthropology and Education Quarterly, 2 8(1), 123-126.

Holmes Group. (1990). Tomorrow's schools: Principles for the design of professional development schools. East Lansing, MI: The Holmes Group.

Horowitz, I. L. (1970). Sociological snoopers and journalistic moralizers. Transaction 7, 4-8.

Jones, J. H. (1981). Bad blood. New York: Free Press.

Journal of Teacher Education. (1991). The ethical responsibilities of teaching [Entire issue], 42, 162-239.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills.: Sage.

Lincoln, Y. S., & Guba, E. G. (1989). Ethics: The failure of positivist science. The Review of Higher Education, 12, 221-240.

Marshall, P. A. (1992). Research ethics in applied anthropology. IRB: A Review of Human Subjects Research 14, 1-5.

McElroy, L. (1990). Becoming real: An ethic at the heart of action research. Theory into Practice, 2, 209-213.

Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 6 7, 371-378.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979). The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Washington, DC: Office for Protection from Research Risks, Department of Health and Human Services, U.S. Government.

National Education Association. (1989-1990). NEA handbook. Washington DC: Author.

Patterson, L., Santa, C. M., Short, K. G., & Smith, K. (Eds.). (1993). Teachers are researchers: Reflections and action. Newark, DE: International Reading Association.

Punch, M. (1986). The politics and ethics of fieldwork. Newbury Park, CA: Sage.

Saphier, J. (1982). The knowledge base in teaching: It's here, now! In T. M. Amabile & M. L. Stubbs (Eds.), Psychological research in the classroom (pp. 76-95). Toronto, ON: Pergammon Press.

Sherman, M., & Van Vleet, J. D. (1991). The history of institutional review boards. Regulatory Affairs 3, 615-628.

Shulman, J. H. (1990). Now you see them, now you don't: Anonymity versus visibility in case studies of teachers. Educational Researcher 19, 11-15.

Sockett, H. (1990). Accountability, trust, and ethical codes of practice. In J. I. Goodlad, R.

Soder, & K. A. Sirotnik (Eds.), The moral dimensions of teaching (pp. 224-250). San Francisco: Jossey Bass. Soder, R. (1990). The rhetoric of teacher professionalism. In J. I. Goodlad, R. Soder, & K. A. Sirotnik (Eds.), The moral dimensions of teaching (pp. 35-86). San Francisco: Jossey-Bass.

Strike, K. A., & Soltis, J. F. (1985). The ethics of teaching. New York: Teachers College Press.

Thompson, J. J. et al.(1981). Regulations governing research on human subjects: Academic freedom and the institutional review board. Academe, 25, 358-370.

Thorne, B. (1980). "You still takin' notes?" Fieldwork and problems of informed consent. Social Problems, 27, 284-297.

Wax, M. L. (1980). Paradoxes of "consent" to the practice of fieldwork. Social Problems, 27, 272-283.

Wax, M. L., & Cassell, J. (Eds.). (1979). Federal regulations: Ethical issues and social research. Boulder, CO: Westview Press.

Wilson, S. M. (1995). Not tension but intention: A response to Wong's analysis of the researcher/teacher. Educational Researcher, 24(8), 19-22.

Wong, E. D. (1995a). Challenges confronting the researcher/teacher: Conflicts of purpose and conduct. Educational Researcher, 24(3), 22-28.

Wong, E. D. (1995b). Challenges confronting the researcher/teacher: A rejoinder to Wilson. Educational Researcher, 24(8), 22-23.

Yin, R. K. (1989). Case study research: Designs and methods. (rev. ed.). Newbury Park, CA: Sage.

Cite This Article as: Teachers College Record Volume 99 Number 2, 1997, p. 247-265

http://www.tcrecord.org ID Number: 10256, Date Accessed: 8/8/2006 2:05:40 PM

The Use of Qualitative Methods in Large-Scale Evaluation: Improving the Quality of the Evaluation and the Meaningfulness of the Findings - by Julie Slayton & Lorena Llosa — 2005

In light of the current debate over the meaning of scientifically based research, we argue that qualitative methods should be an essential part of large-scale program evaluations if program effectiveness is to be determined and understood. This article chronicles the challenges involved in incorporating qualitative methods into the large-scale evaluation of the Waterford Early Reading Program and the decisions made in light of those challenges. We demonstrate that, in spite of the challenges, there are significant benefits associated with using qualitative methods on a large scale. More specifically, by using qualitative methods, we were able to improve the evaluation and generate findings that were meaningful and useful to stakeholders. It is our hope that our experiences can be used by others to inform their work as they seek to incorporate qualitative methods into their large-scale evaluations.

 INTRODUCTION

The No Child Left Behind Act of 2001(NCLB) has prompted educational researchers to conduct scientifically based research to inform instructional practices and support the adoption of instructional programs. It has also prompted educational researchers to debate what is meant by the term scientifically based research and what methodologies are legitimate, appropriate, and in line with such research efforts (Eisenhart & Towne, 2003). The most prominent interpretation of the legislation is that scientifically based research is limited to experimental or quasi-experimental designs (Slavin, 2002) and quantitative methodologies. Others argue that scientifically based research should offer opportunities to engage in a variety of designs and methods, allowing the research question to drive the design and methods adopted (Burkhardt & Schoenfeld, 2003; Chatterji, 2004; Erickson & Gutierrez, 2002; Feurer, Towne, & Shavelson, 2002).

In fact, Maxwell (2004) made a strong case for not limiting scientifically based research to experimental and other quantitative methods. He argued that ‘‘qualitative research is a rigorous means of investigating causality'' (p. 3) and should be an integral part of scientifically based research in education. We agree with Maxwell—that to understand study outcomes, quantitative methods are insufficient. In program evaluation, for example, when no evidence of effectiveness is found using quantitative methods, qualitative methods can explain why the program was not successful. These explanations might include issues such as the following: The materials might be too difficult to implement, the teacher might not be implementing the program appropriately, the program might not be engaging and the students do not pay attention, or the professional development is insufficient to support the program implementation. When a program is determined to be effective, qualitative methods can confirm that it is actually the program that is responsible for the effect. Qualitative methods pose the questions of how and why a program does or does not cause the intended effect and not simply whether a program causes the intended effect. Therefore, we believe that qualitative methods should be an essential part of large-scale program evaluations if program effectiveness is to be determined and understood.

The purpose of this article is to document the challenges and benefits associated with a large-scale evaluation that incorporated qualitative methods. We used qualitative methods in order to gain a ‘‘greater program understanding and more explanatory power, specifically about why and how certain outcomes were attained or not'' (Greene, 1998, 141). Below, we describe the evaluation. We chronicle the challenges we faced and the decisions we made in light of those challenges. Finally, we document the benefits of using qualitative methods to inform the evaluation itself and generate findings that were meaningful and useful to stakeholders. We hope that our experiences can inform others who are interested in or actively attempting to incorporate qualitative methods into their large-scale evaluations.

BACKGROUND

In 2001, the Program Evaluation and Research Branch was asked to evaluate a computer-based reading program adopted by the Los Angeles Unified School District (LAUSD) to supplement an existing reading program in kindergarten and first-grade classrooms. The district had adopted the Waterford Early Reading Program (Waterford program) in 244 schools and 2,235 classrooms, to be used with 81,000 students in kindergarten and first grade. Our branch was already responsible for evaluating the implementation of the district's primary reading program, Open Court Reading (Open Court). Now, we were being asked to examine the extent to which this new program, Waterford, impacted student achievement and teacher practice when it was used as a supplement to the Open Court Reading program. In kindergarten, each child was expected to spend 15 minutes a day working individually at one of the three Waterford computers placed in the classroom for that purpose. In first grade, children were expected to spend 30 minutes a day at one of the three computers with the courseware. The program's courseware could be personalized for each child's learning pace and reading level, and the program could provide teachers up-to-date information on their students' progress and needs as they moved through the various reading levels. In addition, children had a series of books and videotapes to take home, thus giving them further exposure to the material and giving parents the opportunity to read with their children.

RESEARCH QUESTIONS AND STUDY DESIGN

Over the course of the 3-year evaluation period, we attempted to address a range of research questions. The two overarching research questions we addressed were as follows:

1. Does the Waterford Early Reading Program, as implemented in the LAUSD, have an effect on student reading achievement?

2. To what extent is the Waterford courseware being implemented?

To address the research questions, we had no choice but to adopt a quasi-experimental design. One of the realities of program adoptions in school districts is that these decisions are driven by district policy and not by evaluators interested in determining the effect of a particular program or intervention. As a result, evaluators, by and large, cannot control how students, teachers, and schools are assigned to receive a given program or intervention. This reality makes experimental design nearly impossible and forces evaluators to use quasi-experimental design at best, and more often, one-group designs. In this case, we were fortunate that the district did not choose to implement the program districtwide, but only in a subset of schools meeting a certain set of criteria. As a result, we were able to identify a comparable group among those schools that were not receiving the Waterford program and use a quasi-experimental design. The sample of our study consisted of 2,000 students in 100 kindergarten and 100 first-grade classrooms. Thus, in the 50 kindergarten and 50 first-grade treatment classrooms, the Waterford program and Open Court were being implemented. In the other 50 kindergarten and 50 first-grade classrooms, students were only exposed to Open Court.

Our desire to gain a deeper understanding of the program implementation and effect caused us to seek rich narrative data in addition to the test score data necessary to compare the outcomes between the treatment and comparison groups. Therefore, the data collection included extended observations in all 200 classrooms requiring narrative scripting. We also conducted interviews with each teacher and a set of administrators to elicit information regarding teacher use and knowledge of both the Waterford program and Open Court. In addition, we collected quantitative data that included usage data from the Waterford program, individual student test data from our administration of the Woodcock Reading Mastery–Revised (Woodcock, 1998), and the administration of the statewide reading standardized tests.

As will be discussed below, both the requirement to have such a large sample in order to generalize our findings to our very large and diverse district and our desire to incorporate rich narrative data created significant challenges to the overall management of the evaluation in all its stages, particularly qualitative data collection and analysis.

CHALLENGES OF INCORPORATING QUALITATIVE DATA INTO LARGE-SCALE EVALUATION

We were confronted by a number of difficulties in incorporating qualitative data into our evaluation. We will present them in two separate sections. The first focuses on issues related to planning and executing the data collection, and the second focuses on issues related to analysis. In terms of data collection, challenges included overcoming significant time limitations in getting into the field, sufficiently staffing the project, ensuring the quality of the data being collected, guaranteeing confidentiality and anonymity, and managing multiple rounds of data collection. Below we describe each of these obstacles and how we handled them.

DATA COLLECTION

Time Limitations

Time was a significant issue for conducting classroom observations because we wanted to gather information related to initial program implementation and changes in practice/implementation over the course of the school year. We also needed to be able to compare instructional practices in relation to Open Court in treatment and comparison classrooms. Thus, we decided to observe each classroom for 2 days during reading/language arts in both the fall and spring. We needed to conduct our observations within a narrow window of time. Thus, we sought to observe between the 8th and 12th weeks of the school year and then again between the 8th and 4th weeks before the end of the school year. In addition, we had to account for the fact that we operate on a year-round calendar. Some of our schools operate on a traditional calendar, with all students attending from September through June. Others operate with multiple tracks of students from July to June. For example, on a three-track calendar school, for Track A, which begins in September, weeks 8–12 fall in October and November, whereas Track B begins in July, and the 8th week of school is at the end of August. Students then go off track for 2 months, and weeks 9–12 are not until November. Track C, on the other hand, begins in July, and weeks 8–12 fall in August and September. Consequently, we had to adjust to the multiple track schedules, which meant that we had multiple data collection schedules.

Organizational Challenges

In addition to the challenges posed by time constraints, the organizational challenges we encountered to ensure that observations took place within our designated data collection windows were formidable. Although the Program Evaluation and Research Branch staff of 30 researchers is large when compared with other school district evaluation offices, no more than one or two researchers are available to direct an evaluation at any time, and many are responsible for more than one project. As a result, we were dependent on a team of 20–30 individuals who worked for us on a part-time basis to do most of the data collection. Because of the nature of the job— data collection was cyclical and the hours were inconsistent—we were constantly faced with having to recruit and train new data collectors. Because of the substantial focus on a very detailed narrative type of qualitative data collection, which required someone who had the intrinsic strengths of a researcher—inquisitive, detail oriented, organized, focused, flexible, hard working—our preference was to hire graduate students in master's or Ph.D. programs. Although we were surrounded by several major universities with graduate programs, we were competing for these students not only with the graduate schools that employed them as graduate student researchers and teaching assistants but also with other research institutions in the area. With the universities, graduate student researchers received an hourly salary, and at least some part of their tuition was paid for. The students also received health insurance from their schools. Although we could pay a similar hourly rate, we could not provide any benefits to temporary employees. Similarly, we were unable to pay an hourly rate that was competitive with the research organizations in the area. Consequently, our research team was composed of some graduate students and recent college graduates.

In addition to the difficulties we faced hiring staff with the requisite qualifications, our reliance on this large number of transient employees created a host of challenges to data-collection activities. Each round, we were faced with having to replace data collectors because of attrition. In addition, we spent significant amounts of time supervising and mentoring data collectors: talking on the phone, communicating with them via e-mail at least once a week regarding their progress, meeting with them in person, reviewing their work, and answering questions. Dedicating time in this way was critical for us, though, because we depended on data collectors to get the data we needed to conduct an effective and comprehensive evaluation.

Protocol Development

Prior to entering the field, we discussed the types of data that we would need to collect in order to answer our research questions. We knew that we wanted as much information as we could gather about each classroom in the context of reading/language arts instruction, particularly student use of the Waterford courseware, teacher pedagogy, and student behavior. Others have used checklists, responses to directed questions, summaries of classroom activities, and ratings (Aschbacher, 1999; Datnow & Yonezawa, 2004; Noga, 2002; Piburn & Sawada, 2001; Saxe, Gearhart, & Seltzer, 1999), but we decided that they were they were limited for a variety of reasons. Checklists required predetermined categories, and we did not know enough about what we should expect to see in the classroom. In addition, checklists required that data collectors be knowledgeable about the courseware, the Open Court curriculum, and reading/language arts instructional practice in order to understand the established categories. It is difficult to establish consistency with checklists and summaries because each data collector is coding the data while collecting them, which requires a great deal of content expertise. More important, because data reduction takes place at the point of data collection when using a checklist or summary, the data can be used for quantitative analysis but not for in-depth qualitative analysis. Consequently, we knew that we did not want to use a checklist or a summary-type protocol. Instead, we turned to open-ended narrative-based protocols to overcome the limitations of checklists and summaries.

We felt that narrative-based observation protocols would remove the responsibility for making coding decisions from the data collector. His or her role would be to record the observation with sufficient detail to allow us to code and analyze the data. Additionally, a narrative-based observation would capture data in a form that could be used for both qualitative and quantitative analysis. There would be sufficient information to quantify those items that would be captured in a checklist and to provide the details and nuances of teacher pedagogy and student behavior that would be essential to understanding the reasons behind the evaluation outcomes and that could inform future practice and program implementation. We planned on creating narrative-based protocols that would require data collectors to write field notes. We defined field notes as a written narrative describing, in concrete terms and in great detail, the duration and nature of the activities and interactions observed. Classroom data collectors would become the ‘‘eyes and ears'' of the project, and their notes would describe the overall context in which reading/language arts instruction took place.

We designed one narrative-based observation protocol to capture data about the use of the Waterford courseware and a separate narrative-based observation protocol to collect data regarding teacher practice and student behavior in the context of Open Court instruction. The rationale for a protocol capturing teacher practice and student behavior was that we would be able to compare our treatment and comparison classrooms and thus separate what students were learning during Open Court instruction from what they were learning from the Waterford courseware.

The Waterford computer log was designed to capture information from three students simultaneously as they each worked individually on the courseware at a computer station. We needed the protocol to record the students' names, the time they began using the courseware, and the time they completed their turn, in addition to a detailed narrative about the students' behavior while using the courseware. The name and time information would be necessary to relate students' time spent on the computer with their test scores. The students' names and the narrative would document the students' level of engagement with the courseware, anything that might impact their turn (i.e., computer problems and interruptions), and the extent to which they fully used the program. We could also use this data to examine whether there was a relationship between level of engagement and test scores. Figure 1 shows an excerpt from our Waterford computer log for one student. The full protocol contains room for three students per page.

The classroom observation protocol was designed to document classroom interactions during reading/language arts. The prompt provided to data collectors is presented in Figure 2. Pages of lined paper follow the prompt.

In addition to the observation protocols, we felt that we would need a supplemental protocol for data collectors to gather some initial information regarding the classes they were to observe. Thus, we crafted a one-page initial interview intended to take no more than 5 minutes. In treatment classrooms, we wanted to know about teacher training on the Waterford courseware, computer installation, whether all the equipment was there

Teacher Code: Location Code: Date:

Grade: Observer:

COMPUTER STATION 1: Name: _________ Time In: _______ Time Out: ______

Figure 1. Waterford Computer Log

and working, and whether teachers were using materials intended to support courseware use in the students' homes. For both treatment and comparison classrooms, we also gathered general information about the classroom, such as students' English language development levels, the teacher's lesson goals and objectives for the day, and the Open Court unit and lesson that they would be teaching. The challenge in developing this protocol was to make it comprehensive, yet brief enough to ensure that teachers would give their time to answer the questions before class started for the day.

Finally, we wanted to ensure that our data collectors would have the opportunity to record any information or data gathered during their observation that did not belong on any of the other observation protocols. This additional protocol would also be a place for data collectors to document their experiences, biases, and likes and dislikes of a classroom observation experience. It would allow the data collector to intentionally place any subjective comments they had regarding their observation so that they could avoid expressing these comments within the context of the objective field notes taken during the observations. Thus, we designed a reflective note protocol that asked the data collector to reflect on the two days of observation.

In addition to classroom practice, we wanted to capture information related to teachers' experiences using the Waterford program and Open

In a narrative, describe the activities taking place in the classroom. Focus on teacher-to- student(s) and student-to-student interaction, as well as activities of individual students. A narrative requires that you directly quote the teacher as he or she asks questions, models reading strategies, presents material, and works with students in a whole group, small groups, or individually. Additionally, you should document student engagement in response to the teacher and individual or group work and nonwork activities.

Figure 2. Classroom Observation Protocol Prompt

Court during reading/language arts instruction. We designed one interview protocol for treatment teachers and one for comparison teachers. Each interview took approximately 30 minutes to administer. We asked both comparison and treatment teachers questions regarding the Open Court training and coaching that they received, their perceptions about the effectiveness of Open Court, and their implementation of Open Court. For treatment teachers, we also asked an additional set of questions specifically geared toward the Waterford program implementation. The topics of the interviews included training, frequency of use of the courseware and supplementary materials, the way that the teacher used Waterford during Open Court instruction, whether the courseware fully engaged students at all times, the features of the program, and perceptions of the effectiveness of the courseware.

Training the Data Collector

Given the demanding nature of our protocols, it was clear to us that we would have to extensively train our data collectors to ensure the quality of the data collected. Moreover, because we would have to observe 200 classrooms within a period of 2–3 months, our challenge was to train a sufficient number of data collectors to gather high-quality, detailed data. Training thus became a critical component of the evaluation. This was also an extremely time-consuming activity; we had to dedicate 3 to 4 full days to training. Because we had a fall and a spring data collection and multiple tracks for each round, we had to conduct four observation training sessions per year.

During the first day of training, an overview of the evaluation project was presented, and the data collectors were introduced to the protocols. Additionally, information was provided regarding the procedures for contacting schools and scheduling observations. The second day consisted of a mock observation. Data collectors, accompanied by one of us, observed a class for an hour and were then given feedback on their notes. On the third day, the data collectors worked on several activities to reinforce the strategies for documenting their observation. In addition, we shared with data collectors some of the coding and analysis that would be done using the data that they collected. In the spring, we included interview training because data collectors were also responsible for interviewing teachers at each school in which they conducted an observation. For spring observations, we required all returning data collectors to participate in a 1-day retraining session during which they were given the opportunity to self-critique their observation notes from the fall.

After training, we gave each data collector one classroom to observe. We required that they return to us so that we could review their field notes after completion of the first observation. This allowed us to monitor and provide feedback to ensure that the data we were collecting met our expectations. In some cases, this led to additional meetings with the data collectors. We learned that this level of attention to training data collectors was critical to our ability to collect high-quality, detailed observation data that focused on the interactional dynamics of the classroom situation—that is, the ‘‘immediate environment of learning'' (Erickson, 1988)—and link these dynamics to student achievement.

Confidentiality, Anonymity, and Building Teacher and Administrator Trust

We were presented with a series of obstacles as we entered schools to observe classrooms. At a time when schools found themselves the subject of a multitude of evaluations from grant providers, the state, federal programs, and our district office, many principals would have preferred to exclude their schools from our evaluations because of the burden that they believed our presence placed on their teachers. Even though teachers and principals were required to participate, we still had to persuade them to allow us into their classrooms and schools.

In the correspondence we sent out, we informed teachers and administrators that our sole purpose in their schools and classrooms was to observe program implementation and not to evaluate a particular teacher or school. However, it was often difficult for teachers and administrators to understand the distinction that we were making. Although we explicitly explained that our purpose in the schools and classrooms was separate from any type of teacher evaluation, and that we did not use teacher or school names when reporting findings, teachers often felt threatened by our presence in their classrooms. The impact of this tension on our ability to collect data was that we often found ourselves talking to teachers and principals to explain our purpose in their schools and assuage their reservations and concerns. This activity consumed a great deal of our time, and in some cases, it required that we go to the schools to conduct observations if a data collector was unable to gain access. It was important that every classroom selected for the study actually participate because allowing teachers and administrators who did not wish to participate in the study to opt out compromised the generalizability of the study and, more broadly, the branch's ability to conduct any future evaluation of districtwide program implementations.

An obstacle that we needed to overcome in the Waterford program evaluation was explaining to teachers in comparison classrooms why they had been selected to participate in an evaluation of a program that they did not have. In fact, in some cases, administrators voiced concern that participation as a comparison school might exclude them from receiving the program in the future. Again, we spent time during each round of data collection talking to principals and teachers to assure them that their participation would in no way impact whether their school would receive the Waterford program in the future, and that they were selected as comparison schools so that we might determine whether the Waterford program was impacting student achievement above and beyond the impact of Open Court.

CHALLENGES IN QUALITATIVE DATA ANALYSIS

A significant challenge presented by the size and scope of the Waterford Evaluation design was the use of traditional qualitative methodologies to collect and analyze data about program implementation within a large-scale multisite study. Because the Waterford program is a supplemental program, we needed to understand both the extent to which it was being implemented as a reading program independent of anything else happening in the classroom and how it was implemented in relation to Open Court. Moreover, because the design of the study was quasi-experimental, we needed to be able to assess the quality of implementation of Open Court in the comparison classrooms as well. The primary difficulties related to analysis were that we had thousands of pages of qualitative data to analyze— detailed narrative Waterford program use notes collected using the computer log protocol for 100 classrooms over 4 days and detailed narrative Open Court notes collected using the classroom observation protocol for 200 classrooms over 4 days—and all of our analysis had to be completed in less than 5 months so that we could provide a report to the Board of Education by December.

Within the analysis, many challenges presented themselves. The qualitative data analysis had to focus on a number of separate but related sets of data. In treatment classrooms, the analytic focus included (1) teacher pedagogy and its relationship to the implementation of the Waterford program and Open Court and (2) student behavior in relation to the Waterford program and teacher pedagogy. In comparison classrooms, the analytic focus was limited to teacher pedagogy in relation to the implementation of Open Court and student behavior in relation to teacher pedagogy. To determine the extent to which the Waterford program was responsible for increased reading ability, we had to be able to disentangle what students were learning through their usage of the Waterford program from what they were learning from the teacher and Open Court.

Thus, we adopted a combination of strategies to accomplish our goal. To be able to analyze the tremendous amount of data collected, we sought to create a research team similar to those typically found at graduate schools employing graduate student researchers led by a primary investigator. Thus, we hired six graduate students to work closely with us on data coding and initial analysis of the Waterford program data and the data on teacher-directed Open Court instruction. Each member of the team was responsible for coding a grade level and track for both fall and spring rounds of data collection and then for both treatment and comparison classrooms. Our general approach to qualitative data analysis was to use inductive analysis (Patton, 2002). As we moved forward through the coding process, we met weekly to identify themes and categories, after which each person went back and reexamined his or her data to verify the presence or absence of a given theme or category and to ensure consistency in the process. We began with specific observations of teacher and student behavior and built toward general patterns of behavior.

WATERFORD PROGRAM DATA CODING AND ANALYSIS

For the Waterford program data, the team of analysts examined notes for each student who used the courseware in our 100 treatment classrooms. The notes provided sufficient detail for the analyst to determine the extent to which the student was engaged. As can be seen in the following examples, observation notes captured a range of behaviors by students during their use of the Waterford courseware. We were generally able to tell if a student was highly engaged, following the directions of the courseware (Example 1); distracted by other students (Example 2); or interrupting other students working on other computers or in large group activities with the teacher (Example 3).

Example 1

Maria puts earphones on. Very engaged. Works diligently. Takes the microphone and speaks into it. Prints something out. There's a chicken with eggs on the screen. She puts the printout next to her, continues to work. Prints again. Letter F, D, R on the screen. Follows the ‘‘X'' with her finger on the screen. She's making letter sounds. Is very engaged!

Example 2

She is talking to Joy. Fighting with Joy. Made up. Talking to Joy about what's on her screen. Headphones off. Not watching. Recess. Participating in lesson. Not looking at computer. Watching Joy's screen. Talking to Joy re: her screen. Talking but not about lesson. Waiting on computer screen.

Example 3

Jacinta watches photos of city and country. Distracted by Nikki's singing. The City Mouse and the Country Mouse, long story. She pays close attention to screen the whole time. Now follow-up activities for story. She says ‘‘Games!'' Word puzzle, parrot picture underneath. Keeps saying ‘‘Awesome'' as picture is revealed. Distracts Nikki and another student. Likes the games and gets excited. Sings aloud. Distracts Alan and another girl. ‘‘ed'' on screen. Jacinta is singing to a song—loud. Teacher says to her, ‘‘Shhh.'' Jacinta is very into songs. Aide and Teacher tell her ‘‘Shhh.'' ‘‘Whale, wh, wh,'' song. Sings aloud. Jacinta ignores Nikki—sings and sings.

The detail present in our data allowed us to create a rubric that defined different levels of student engagement. The rubric was a 4-point scale ranging from fully engaged to completely off-task. We then were able to apply that rubric to make determinations about the level of engagement, the reasons behind that level, and other factors that affected students' turns at the computer (interruptions, computer problems, Open Court instruction).

While we were working on the individual student level analysis, the 4 days of data for each classroom allowed for classroom-level themes to emerge, such as computer problems and how they affected the use of the courseware, the relationship between student level of engagement during Waterford program usage and during Open Court instruction, and English learners and their engagement with the courseware as compared with their engagement during large group instruction and English-only students.

OPEN COURT DATA CODING AND ANALYSIS

For the Open Court data, the team examined notes for each reading/language arts activity. The notes provided sufficient detail for the analyst to examine in greater depth the themes and categories identified during the initial coding. Below is an excerpt from field notes taken during a kindergarten ‘‘Preparing to Read'' lesson:

T: J is the magic letter. We are going to learn the poem for the letter J. All right. Her name, her name is Jenny. I have a friend named Jenny!

Ss: Jennifer!

The teacher demonstrates juggling.

T: I tried to juggle at home with oranges. But you know what, they got squashed.

The teacher introduces the /j/ sound.

T: Softly /j/ . . . /j/ . . . /j/ . . . Let's see if I get this sound right.

The teacher reads the poem to the students and then turns on a tape of a song that says ‘‘the J sound goes /j/ /j/ /j/.''

  She now introduces the Jj card pictures. She holds up cards with the following words: jar, judge, juice, jellyfish, and Jello. The teacher describes each word and gives a Spanish translation.

The teacher now passes out Jj letter cards.

T: Is it a fan?

Ss: No.

T: Do you put it in your nose? I'm gonna say some words and if they begin with the /j/ sound, you lift it.

T: Green.

Some students lift their cards.

T: Did it say jreen?

The teacher goes through additional words, repeating the same pattern.

T: Now for my magic bag. I'm thinking of something that I drink in the morning.

S: Orange juice.

T: Yes, it can be apple juice, grapefruit juice, or prune juice because one day I'll be old and need prune juice.

The teacher is now asking questions asking students to identify j words she pulls from her bag. She takes out a box of Jello. She spells out Jello in a humorous way.

Ss: Gelatina!

T: I'm going to put the container like this so you can see the letter j in

Jello.

T: Jelly beans and the company that makes them are called Jolly

Ranchers. Jolly means happy. He is a happy man.

The teacher takes out a jump rope.

T: This is a jump rope. I trained with the best. The rope needs to jump. I think I know what to do. But you have to help. Every time the rope touches the ground you have to go /j/.

The teacher starts jumping rope. Students say jah.

T: Not jah. Its /j/, softly. I'm gonna do it one more time, but my heart . . . I think that's about it. Go back to your usual spots on the rug.

The teacher approaches the students individually. Each student says j.

T: I could have had a cardiac arrest today, but it would have been worth it. You learned it!

The observation notes, like those above, were examined in relation to the Open Court teachers' manuals and relevant research literature to assess the quality of the implementation and teacher pedagogy. For example, for the data presented above, the teacher's manual instructs the teacher to introduce the Jj sound as follows:

•

Display the Jj Alphabet Card and say the sound of the letter, /j/. Show the picture for the /j/ sound and teach the short poem for /j/:

Jenny and Jackson like to have fun.

They play jacks, jump rope, and juggle in the sun.

Each time they jump, their feet hit the ground.

/j/ /j/ /j/ /j/ /j/ is the jumping-rope sound.

•

Repeat the poem, emphasizing the initial /j/.

Next, the lesson directs the teacher to have the students listen for the initial /j/ sound:

•

Hold up and name each of these Picture Cards: jam, jar, judge, jeans, juice, and jellyfish. Ask the children to listen for the /j/ sound at the beginning of the words.

•

Give each child a Jj Letter Card.

•

Have the children hold up their Jj cards each time they hear the /j/ sound at the beginning of the words. Try these words:

Green jail Jeans Gail Jill

Jake Jam Jim gas

In the next section, the teacher was directed to play the ‘‘I'm Thinking of Something that Starts with ___'' game. According to the teacher's manual, the teacher should:

•

Play the ‘‘I'm Thinking of Something that Starts with ______'' game, using words that begin with /j/. Choose objects that are outside of the room but give the children some clues to what you are thinking of. You might try the following objects and clues:

Something you drink in the morning (juice)

Something you put on your toast (jam or jelly)

The sound bells make (jingle)

•

If you have children in your class whose names begin with /j/, you might want to use their names in the game.

After analyzing a significant portion of the notes and determining themes and categories by comparing the notes with the teacher's manual, we created a rubric to assess the quality of pedagogy combined with the fidelity of program implementation across the 200 classrooms of data. A 5-point scale was constructed to reflect high- to low-quality pedagogy and program fidelity (Slayton & Llosa, 2002). Our definition of high-quality pedagogy was developed after an extensive review of the research literature on reading and comprehension instruction (Adams, 1990; Duffy, Roehler, & Herrmann, 1988; Ehri, 1992; Yopp, 1992; Pearson, 1984; Pressley et al., 1992). In this way, data from all four days of observation in the 200 classrooms were examined by grade and by track for overall findings related to implementation and pedagogy.

The types of analyses we undertook were labor- and time-intensive. However, they were intended to provide decision makers with concrete information to understand the quality of implementation and pedagogy and student engagement. In addition, we used the rubric scores and the ratings of student engagement as variables in our quantitative analyses of program effect (Slayton & Llosa, 2002). The connections between achievement and pedagogy or implementation would allow us to examine the role of the teacher in impacting program implementation and student achievement.

BENEFITS OF QUALITATIVE DATA AND ANALYSIS

Our choice to incorporate rich narrative observation data from the outset of the evaluation had a number of benefits. By observing teaching practice and program implementation from the beginning, we were able to gain a thorough understanding of the context in which the program was being implemented, including the quality of the pedagogy provided by teachers, the range of student behavior while using the program, and the overlap between Open Court and the Waterford program during reading/language arts instructional time. One benefit was that we were able to eliminate potential explanations for differences between the treatment and comparison groups and make thoughtful changes to the evaluation plan to redistribute resources and narrow the focus of the evaluation to those elements that were most likely to be responsible for program effect. A second benefit of using these rich narrative observation data was that they added significantly to the usefulness of the findings for policy-making purposes.

TO IMPROVE THE EVALUATION

As we discussed earlier, when the evaluation began, one of our concerns was the supplementary nature of the Waterford program. To determine whether the Waterford program was responsible for any benefits beyond those resulting from the primary instruction, we first had to establish that the quality of pedagogy was comparable in treatment and comparison classrooms. Thus, in the first year, we observed treatment and comparison classrooms for two days in the fall and the spring. We found no differences in the quality of pedagogy in kindergarten and first-grade treatment and comparison classrooms (Slayton & Llosa, 2002). This assured us that any differences in achievement that we might find would be the result of the use of the Waterford program. This finding also allowed us to eliminate quality of pedagogy as a primary focus for future data collection and analysis.

We used our experience in the first year to make changes to our data collection and analysis in the second year of the evaluation. We turned our attention to a more careful examination of the treatment classrooms in terms of students' level of engagement and the interaction between the primary reading program and the Waterford program. To address these issues more directly, we revised our Waterford computer log (see Figure 3). In addition to the space provided for narrative notes, we included the ratings of engagement that we developed in the first year. These ratings ranged from fully engaged to off-task. We also provided data collectors with a space to identify the type of activity—OC Green Section, OC Red Section, OC Blue Section, or other—under way in the classroom during each student's turn on the Waterford program. These activities refer to different components of the Open Court program. The Open Court 2000 Teacher Edition is divided into three separate components: Preparing to Read, Reading and Responding, and Integrating the Curriculum. Each section is color-coded. The Preparing to Read component is green and is often referred to as the Green Section. This section focuses on decoding and

Teacher Code: Location Code: Date:

Grade: Observer:

COMPUTER STATION 1: Name: _________ Time In: _______ Time Out: ______

o

Fully engaged o

Minor distractions o

Distracted o

Off-task

Classroom activities: o

OC Green Section o

OC Red Section o

OC Blue Section o

Other: ______

Figure 3. Revised Waterford Computer Log

fluency. The Reading and Responding component is color-coded red and is referred to as the Red Section. This component focuses on reading comprehension instruction. The Integrating the Curriculum component is color-coded blue and is referred to as the Blue Section. This component focuses on language arts and writing.

These changes to the protocol also required changes in our data-collector training. These changes took two forms. The first was to train our data collectors to be able to characterize each student's level of engagement as fully engaged, minor distractions, distracted, or off-task. This understanding would allow them to check the appropriate box while conducting the observation. In addition, we had to provide additional training for the narrative note-taking. We focused more attention on the process of documenting the specific interaction between the student and the computer. In other words, we trained our data collectors not to characterize student behavior as engaged or on-task, but instead to carefully describe the specific actions in which students were engaged while sitting in front of the computer. For example, a student might trace the letters on the computer screen, sing along with a song, stick a pencil in the disk drive, point to a neighbor's screen, or look away from the computer. In the first year, while data collectors attempted to capture these behaviors, they would sometimes summarize the student's behavior as being engaged or being off-task (see Example 1 presented previously). These overgeneralizations of student behavior as engaged or off-task made it difficult to ensure that our interpretations of the students' levels of engagement were reliable across data collectors during the first-year data analysis phase. At the time, we also did not want the data collector coding student engagement because we did not yet have a common set of definitions for engagement and had no realistic understanding of what student engagement with the program would look like until we had observed it. Instead, we wanted narrative descriptive notes to allow the analysis team to develop a coding scheme that would be applied consistently across all observations. In the second year, on the other hand, we did have a very clear understanding of the range of student behaviors while interacting with the program. Therefore, we felt comfortable adding the ratings to the protocol.

Our findings in the second year also shaped our third-year data-collection plans. In the second year of data collection and analysis, we also found no differences in the quality of pedagogy in treatment and comparison classrooms. Consequently, we decided not to collect classroom observation data from comparison classrooms in the third year. Additionally, in the first two years, we found no changes in pedagogy from the fall to spring rounds, so we also made the decision not to conduct two rounds of classroom observations, but to only observe treatment classrooms in the spring. Because we did not want to lose representativeness in the teacher use of the program, we did one 3-day observation instead of the two 2-day observations that we had conducted during the previous 2 years.

TO PROVIDE MEANINGFUL FINDINGS AND INFORM POLICY

In addition to benefiting the evaluation process, the incorporation of rich narrative qualitative data added significantly to our ability to interpret the findings. Through the quantitative analyses, we found no differences in reading achievement between students exposed to the Waterford program and those who were not. This was true in the first and second years of the evaluation (Hansen, Llosa, & Slayton, 2004; Slayton & Llosa, 2002). The narrative observation data allowed us to explore the possible explanations for our findings. We were able to consider whether varying qualities of teacher pedagogy or implementation of Open Court in treatment and comparison classrooms were responsible for our results. We also considered whether the varying levels of student engagement both during primary instruction and during Waterford program use explained the findings. Finally, we examined the possibility that the reason why we found no differences was that the Waterford program was being used to supplant rather than supplement primary Open Court instructional time that focused on developing students' knowledge of sounds and letters, decoding, and fluency. We will discuss each of these in turn and explain how our findings allowed us to make specific recommendations that were used by the district to improve the district's elementary reading plan implementation.

First, the rich qualitative data allowed us to determine that there were no differences in the implementation of Open Court or the quality of pedagogy provided by teachers to students in both treatment and comparison classrooms. This allowed us to confirm that our two groups—treatment and comparison—were comparable in terms of the primary instruction that they were receiving. This, combined with the fact that student achievement was also the same, indicated that the Waterford program had no added benefit for students in treatment classrooms. Furthermore, the qualitative data allowed us to determine that the quality of pedagogy and Open Court implementation in both treatment and comparison classrooms was low. We were able to demonstrate the specific ways in which teachers did not provide sufficient primary reading/language arts instruction to their students by including in our report verbatim text of classroom observations. We then made recommendations regarding how the district might improve the implementation of its primary reading program, Open Court, in addition to any recommendations we made that were directly related to the implementation of the Waterford program. The district responded to this finding by increasing its focus on improving Open Court implementation through a variety of professional development mechanisms.

Second, the rich qualitative narratives allowed us to determine that student engagement fluctuated both during teacher-directed instruction and during Waterford program time. Again, this led us to conclude that the students' instructional experiences were comparable in the two groups. Students were as likely to be disengaged when using the Waterford program as they were during Open Court time, thus demonstrating that the use of technology for instruction did not guarantee that students would be engaged. This finding allowed us to make a number of recommendations to improve the implementation of the Waterford program. We suggested that the computers be positioned within the classroom in such a way as to reduce the likelihood that they would distract students during their Open Court instructional time and to increase the likelihood that students using the computers would not be distracted by each other or their classmates. We also recommended that the computers be moved to a lab so that all students could work on the Waterford program at the same time with teacher supervision.

Third, we were able to use the detailed narrative to determine that the Waterford program was supplanting the Open Court instructional time instead of supplementing reading/language arts time. During the first year, we noticed that there was an instructional time overlap between the time students spent using the Waterford program and Open Court Green Section instruction. As mentioned earlier, we made changes to the computer log protocol to specifically examine the extent of this overlap. The data gathered during the second year allowed us to determine that the length of the reading/language arts block and the policy governing the use of Open Court and the Waterford program

put teachers in the position of having to either 1) use the supplementary reading program during the primary reading program instruction; 2) use the Waterford program during other portions of the instructional day (e.g., during math instruction); 3) or not have all of their students use the Waterford program on every day. These time constraints also do not take into account other events that interfere with typical daily instruction like [professional development time], assemblies, parent teacher conference weeks, testing schedules, and a host of other events that further limit[ed] the amount of time available during the regular instructional day. Consequently, it was not surprising to find that approximately half of the kindergarten classrooms and two-thirds of the first grade classrooms were observed using the [Waterford program] during Green Section instruction. In other words, on any given day, between 20–31% of students missed all or part of their primary phonics instruction and instead were exposed to the Waterford courseware, the supplementary reading program. Thus, the program was not used to supplement the primary reading program. (Hansen et al., 2004, p. 64)

Our recommendations included limiting the use of the Waterford program to those students who demonstrated the need for additional support; using the Waterford program only with students who needed additional support during the existing intervention programs that took place after school, on weekends, and during the summers; and extending the school day so that all students would be able to use the program on a daily basis for the recommended amount of time. The district responded to these recommendations by providing schools and teachers with new guidelines regarding the use of the Waterford program during instructional time. In addition, the findings provided support for the district's decisions to move to an all-day kindergarten schedule and to not expand the implementation of the Waterford program beyond its initial schools.

CONCLUSIONS

In 2001, the federal government stepped into the debate over what is to be considered quality research. In the No Child Left Behind Act, Congress set forth the expectation that for research to be considered of a high quality, it must be scientifically based. The use of this term, scientifically based research, has created quite a stir in the research and evaluation communities. It has lead to an ongoing public discussion of what is meant by scientifically based research. Much of the debate has swirled around whether scientifically based research demands the use of experimental or quasi-experimental design. A second and equally important discussion is under way concerning the types of research methods that are to be considered legitimate, appropriate, and in line with scientifically based research designs. We argue, as others have, that in addition to the use of quantitative methods, scientifically based research designs should employ qualitative methods when the goal of the research is to evaluate program effectiveness. We believe that qualitative methods have the power to explain why and how program outcomes were or were not attained. Thus, the purpose of this article is to document the challenges and benefits associated with a large-scale evaluation that incorporated qualitative methods. We chronicled the challenges we faced and the decisions we made in light of those challenges. We also demonstrated that, in spite of the difficulties associated with using qualitative methods on a large scale, there are significant benefits as well. More specifically, by using qualitative methods, we were able to improve the evaluation and generate findings that were meaningful and useful to stakeholders. It is our hope that our experiences can be used by others to inform their work as they seek to incorporate qualitative methods into their large-scale evaluations.

References

Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press.

Aschbacher, P. R. (1999). Developing indicators of classroom practice to monitor and support school reform (CSE Technical Report 513). National Center for Research on Evaluation, Standards, and Student Testing (CRESST)/UCLA.

Burkhardt, H., & Schoenfeld, A. H. (2003). Improving educational research: Toward a more useful, more influential, and better-funded enterprise. Educational Researcher, 32(9), 3–14.

Chatterji, M. (2004). Evidence on ‘‘what works'': An argument for extended-term mixed- method (ETMM) evaluation designs. Educational Researcher, 33(9), 3–13.

Datnow, A., & Yonezawa, S. (2004). Observing school restructuring in multilingual, multicultural classrooms: Balancing ethnographic and evaluative approaches. In H. C. Waxman, R. G. Tharp, & R. S. Hilberg (Eds.), Observational research in US classrooms: New approaches for understanding cultural and linguistic diversity (pp. 174–204). Cambridge, England: Cambridge University Press.

Duffy, G., Roehler, L., & Herrmann, B. (1988). Modeling mental processes helps poor readers become strategic readers. Reading Teacher, 40, 514–521.

Ehri, L. C. (1992). Reconceptualizing the development of sight word reading and its relation- ship to recoding. In P. B. Gough, L. C. Ehri, & R. Treiman (Eds.), Reading acquisition (pp. 107–143). Hillsdale, NJ: Erlbaum.

Erickson, F. (1988). Ethnographic description. In J. Von Ulrich Ammon, N. Dittmar, & K. Mattheier (Eds.), Sociolinguistics (pp. 1081–1095). Berlin, Germany: Walter de Gruyter.

Erickson, F., & Gutierrez, K. (2002). Culture, rigor, and science in educational research. Educational Researcher, 31(8), 21–24.

Eisenhart, M., & Towne, L. (2003). Contestation and change in national policy on ‘‘scientifically based'' education research. Educational Researcher, 32(7), 31–39.

Feuer, M. J., Towne, L., & Shavelson, R. (2002). Scientific culture and educational research. Educational Researcher, 31(8), 4–14.

Greene, J. F. (1998). Qualitative, interpretive evaluation. In A. J. Reynolds & H. J. Walberg (Eds.), Evaluation research for educational productivity (pp. 135–154). Greenwich, CT: JAI Press.

Hansen, E. E., Llosa, L., & Slayton, J. (2004). Evaluation of the Waterford Early Reading Program as a supplementary program in the Los Angeles Unified School District 2002–2003 (Planning, Assessment and Research Division Publication No. 170). Program Evaluation and Research Branch, Los Angeles Unified School District, Los Angeles, CA.

Maxwell, J. A. (2004). Causal explanation, qualitative research, and scientific inquiry in Journal item: education. Educational Researcher, 33(2), 3–11.

Noga, J. E. (2002, April). Learning in small classes: Using qualitative methods for evaluation to understand how the process of learning differs in smaller classes (Paper presented at the annual meeting of the American Education Research Association (AERA), LA: New Orleans.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.

Pearson, P. D. (1984). Direct explicit teaching of reading comprehension. In G. Duffy, L. Roehler, & J. Mason (Eds.), Comprehension instruction: Perspectives and suggestions (pp. 222–233). New York: Longman.

Piburn, M., & Sawada, D. (2001).Reformed teaching observation protocol (RTOP): Reference Manual (ACEPT Technical Report IN00–3). Arizona Collaborative for Excellence in the Preparation of Teachers, Tempe, AZ.

Pressley, M., El-Dinary, P., Gaskins, I., Schuder, T., Bergman, J., Almasi, L., et al., (1992). Beyond direct explanation: Transactional instruction of reading comprehension strategies. Elementary School Journal, 92, 511–554.

Saxe, G. B., Gearhart, M., & Seltzer, M. (1999). Relations between classroom practices and student learning in the domain of fractions. Cognition and Instruction, 17, 1–24.

Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21.

Slayton, J., & Llosa, L. (2002). Evaluation of the Waterford Early Reading Program 2001–2002: Implementation and student achievement (Planning, Assessment and Research Division Publication No. 144) Program Evaluation and Research Branch, Los Angeles Unified School District, Los Angeles, CA.

Woodcock, R. W. (1998). Woodcock reading mastery tests: Revised normative update, examiner's manual. Circle Pines, MN: American Guidance Service.

Yopp, H. (1992). Developing phonemic awareness in young children. The Reading Teacher, 45, 696–703.

Cite This Article as: Teachers College Record Volume 107 Number 12, 2005, p. 2543-2565

http://www.tcrecord.org ID Number: 12251, Date Accessed: 8/8/2006 2:02:22 PM

by Robert F. McNergney & Scott R. Imig — October 26, 2006

Like it or not, teacher educators must face the specter of being judged irrelevant if they cannot demonstrate that they add value to the PK-12 educational enterprise. Teachers for a New Era (TNE) is a multi-million dollar, multi-year program intended to shed light on the value added to PK-12 education by colleges and universities via the preparation of teachers. The Carnegie Corporation of New York and both the Ford and Annenberg foundations fund the program (with dollar-for-dollar matching from participating institutions). TNE supports 11 institutions of higher education and their school partners in efforts to determine what, if anything, teacher educators do to help teachers help students learn.1

We use this opportunity, first, to situate our own TNE efforts at the University of Virginia within the larger discussion about teacher education and value-added assessment that is extant at this time. We discuss one model of value-added assessment—its provenance and its unique application at our own institution. Second, we present four critical issues teacher education researchers must address as they engage in value-added assessment. These emanate from within and outside the profession and vary in terms of their prominence in stakeholders' minds. Third, we explicate these issues in terms of particular challenges that have arisen at our own institution—challenges that are as diverse as they are vexing. In doing so, we want to encourage others to scrutinize our work and to offer guidance to teacher education researchers elsewhere who are in similar circumstances.

The research challenge is contained explicitly in Design Principle "A" of the TNE program. Design Principle "A" prescribes that decisions about teaching, learning, and teacher preparation must be driven by "evidence." Dan Fallon, chair of the Education Division of the Carnegie Corporation, and his colleagues have been reluctant to prescribe exactly how evidence should be defined. Yet it is clear that the metric of PK-12 student academic learning, as represented by student test scores, must drive exploration. That is not to say, as we shall note below, that other metrics are out of bounds or irrelevant. Principle "A" is meant to foster the search for knowledge of links between teachers and students, and for that knowledge to guide the assessment and practice of teacher education. All schools, colleges, and departments of education, not just TNE institutions, face increasing pressure to attend to evidence for program legitimacy.

THE CONTEXT OF VALUE-ADDED ASSESSMENT AND TEACHER EDUCATION

In 2002, then-Secretary of Education Rod Paige criticized teacher education and emphasized the need to document the value that it adds to PK-12 public schools. Paige declared education courses to be the "Achilles heel of the certification system." He argued that mandated pedagogical courses "scare off talented individuals while adding little value" (Paige, 2002, p. 40). The invidious comparison catches one's eye, but the key phrase in Paige's comments was "adding little value." In the No Child Left Behind legislation, the value of teaching was to be measured in terms of evidence produced by "scientifically based research." Education was to become an "evidence-based field." If teachers had students who demonstrated academic achievement, then value would be added.

The question of whether teacher effectiveness was a function of participation in a teacher education program loomed large. As the AERA Panel on Research and Teacher Education has demonstrated, teacher education research has not been a particularly fruitful endeavor (Cochran-Smith & Zeichner, 2005). Through the years, neither the funding nor the results achieved by such work have been particularly notable. Yet teachers—who they are and how they are prepared—are routinely characterized as one of the most influential forces in children's lives. With TNE came an unusual opportunity to verify this belief.

A small group of faculty at the University of Virginia worked in the summer of 2002 to construct a conceptual model to guide research on teacher education within the context of value-added assessment (McNergney, Cohen, Hallahan, Kneedler & Luftig, 2002). Mitzel (1960) first envisioned the basic dimensions of the model in Figure 1 nearly a half century ago, and Dunkin and Biddle (1974) elaborated Mitzel's ideas. Figure 1 depicts a set of relationships between and among variables meant to represent the world of research on teacher education, teaching, and learning. Although we describe the steps in the model in terms of our own school of education, they are intended to represent most any university-based teacher education program.

A Model for Teacher Education Research and Evaluation.

Step 1: University students enter the School of Education in their second year from the College of Arts and Sciences and by transferring from other institutions of higher education. The first assessment step is to describe their demographic characteristics such as age, ethnicity, gender, scholastic aptitude, and beliefs about teaching and learning. Effort is devoted to describing students' dispositions, feelings of efficacy, implicit philosophies of teaching and learning, knowledge of disciplines, technology skills, and, their reasons for seeking careers in teaching, particularly when they are "late deciders." Descriptions of the programs in which university students engage, and their performances in these programs, constitute what might be thought of as their training characteristics. Profiles of arriving students permit comparisons to students at other institutions of higher education, to students in other programs at the university, and to one another over time. These profiles form the basis on which the university can track its students longitudinally.

Step 2: Once in education school programs, students demonstrate their abilities to acquire and apply pedagogical content knowledge in their chosen fields. Clinical applications of pedagogical content knowledge, in both live and simulated or case-based settings, provide indications of teaching competence or the effects of teacher education programs. University students' performances yield information on professional teaching processes or teaching performance in controlled settings that is analogous to the kind of information derived from clinical practice in medical education. Here, program faculty can examine how students plan their lessons, how they deliver instruction, and how they evaluate the results of their efforts. These investigations are conducted in different contexts (subject matters, grade levels, schools, and communities).

Step 3: Program faculty study effects that students exert on PK-12 student learning in different contexts. A longitudinal study of our students—following them step by step through their years of preparation and in their initial years as teachers—allows investigators to describe, over time, who students are, how they perform, and what PK-12 students learn. It is also possible to estimate university students' effectiveness using cross-sectional approaches that compare people at different points in their careers. This strategy permits estimates of the relative influence of teachers' characteristics, of their teacher education programs, and of the schools in which they are employed based on PK-12 students' academic test scores.

The bi-directional arrows on the model suggest that the steps are interactive in the sense that they exert reciprocal effects on one another. The arrows also suggest that it is both possible and desirable to retrace steps—from Step 3, to Step 2, to Step 1—to create a programmatic feedback loop. Evidence on PK-12 student learning can be used to inform teaching performance. Investigations of teaching performance, can, in turn, influence selection and preparation of teacher candidates. As Figure 1 suggests, effects between factors may be moderated by teacher and student thinking.

RESEARCH ISSUES AND CHALLENGES

The limitations of conceptualizing and conducting research as we describe it here—that is, in the process-product tradition—were well documented some years ago (Shulman, 1986). Even though studies were conducted in natural settings, critics questioned the external validity of the work. Others drew attention to the raw empirical nature of the approach—the tendency to dwell on teaching behaviors at a molecular level—which tended to diminish the potential explanatory power of theory. Still, others noted what they considered to be an over-reliance on standardized test scores for measuring student success.

Nonetheless, as Shulman observed, the process-product paradigm was remarkably durable. Process-product researchers advanced the conversation on the factors that contributed to school achievement. The approach was consistent with research in behavioristic psychology. Researchers concentrated on teaching and learning in natural settings, not in laboratories. People often perceived the implications for policy and practice as clear and straightforward. And in the end, Shulman argues, process-product research worked—that is, researchers met the aims they set out for themselves.

It is also important to note that while a process-product view suggests a set of critical questions about teacher education, teaching, and learning, the model does not address all the critical issues. Cochran-Smith (2006) and Erickson (2005) note many issues related to teacher preparation that no single strategy (e.g., randomized clinical trials or process-product research) will address. These include, but are not limited to the following:

Are there variations in teacher preparation associated with teachers' retention in hard-to-staff and other schools? What experiences do teacher candidates of color have in mostly White teacher education programs and institutions? Is this important?…What are the school conditions that make it possible for new teachers to take advantage of the resources available to them? How do teachers know if their pupils are learning, and how do they use that information to alter curriculum and instruction?" (Cochran-Smith, 2006, p. 10)

Our own work has been guided by a set of four critical issues that impinge on the quality of the evidence for teacher education program legitimacy (Joint Committee on Standards for Educational Evaluations, 1981). These must be resolved over time if research and evaluation are to shed light on the value teacher education adds to the educational enterprise. We pose them here as questions: (1) Does teacher education research and evaluation produce accurate information? (2) Can research and evaluation produce useful results—results that will guide improvement of the educational enterprise? (3) Is it feasible to conduct teacher education research and evaluation within and across settings? (4) Is it possible to do the kind of teacher education research and evaluation that is deemed informative and also operate within the bounds of propriety? As we explain below, we have only begun to address these issues in our own work.

DOES TEACHER EDUCATION RESEARCH AND EVALUATION PRODUCE ACCURATE INFORMATION?

Our colleague Eric Bredo (in press) has observed that sometimes research and evaluation suffer from the prevalence of Type III Errors. These are failures at the outset to ask the right questions, or those questions that are most germane. Some would argue that accuracy in teacher education program evaluation has been plagued by Type III Errors, leading researchers to seek the wrong kind of information when judging the value of teacher education.

Accuracy means in part that we must consider the value of teacher education in terms of PK-12 student learning. This concern has been reinforced across the nation by the No Child Left Behind legislation and efforts to judge the value of education in terms of PK-12 student learning. The measures of student learning most often of interest, however, do not lend themselves to the assessment of pre-service teacher education programs. To put it another way, the use of students' scores on standardized achievement tests leads us to ask the wrong questions when investigating program efficacy. Why? Pre-service programs exercise no immediate influence on these scores. PK-12 students' standardized achievement test scores may be useful in follow-up studies of teacher education program graduates—or studies of teaching and learning once teachers leave universities and assume full-time employment. But even then, results may be confounded by induction programs, by other efforts aimed at in-service teacher development, and/or by the influences of teacher socialization.

We have tried instead to develop highly focused measures of short-term academic learning, in particular, PK-12 students' mastery of concepts pertinent to data representation and interpretation (statistics), and to use them during the course of pre-service programs. Despite the difficulties of creating and using such measures, they appear to offer reasonable options for assessing the effects of pre-service teachers' training and behavior on student learning.

If PK-12 student learning is the coin of the realm, it is not the only currency in the marketplace. Teacher educators must be judged on factors within their control that will ultimately contribute to student learning. As Figure 1 suggests, there are various ways university teacher education programs influence PK-12 schools, from the moment a student enters the university and eventually assumes a teaching position. Universities attract people with certain qualities into teaching. Teacher education programs provide content knowledge and pedagogical skills to these would-be teachers, who in turn develop a repertoire of strategies and behaviors that enable them to plan, teach, and evaluate their students. Teachers apply what they have learned to engage and motivate students, and ultimately, to enhance students' academic learning. All along the way, this chain of events provides opportunities to gather evidence on the value added to PK-12 education by teacher educators.

To acquire an accurate picture of the value added by teacher education, we must seek answers to a host of questions:

• What are the intellectual, physical, emotional, and social capacities of pre-service teachers that are likely to influence their teaching behavior? How well have they mastered the content they will teach? Do pre-service teachers have the proclivity to continue learning?

• How do teachers in preparation demonstrate teaching behavior that has been shown to relate to or cause PK-12 student learning? Can they plan instruction, implement these plans, and assess their work in ways that are likely to influence students' performances?

• How do pre-service teachers' students think about teaching and learning—are they motivated, challenged, and reinforced by teachers' actions? What do pre-service teachers' students learn?

Can Research And Evaluation Produce Useful Results?

A teacher education program involves a number of audiences whose perspectives on what constitutes "useful" often vary. The audiences include the university student participants and their parents who pay tuition; the PK-12 students these university students teach in field experiences; the parents, teachers, and principals of the schools where the pre-service teachers work; and the teacher educators and university administrators responsible for program operations. State and local education authorities also have a stake in the operation of teacher education programs. As such, these audiences can be expected to influence the kind of questions asked and the type of evidence collected by researchers.

Figure 1 suggests that teacher characteristics (including educational background) affect teaching behavior; teaching behavior, in turn, influences PK-12 student learning. The bi-directional arrows suggest that the steps are interactive in the sense that they exert reciprocal influences on one another. The arrows also suggest that it is possible to retrace steps—from Step 3, to Step 2, to Step 1—to create a programmatic feedback loop. Evidence on PK-12 student learning can be used to inform teaching performance; and investigations of teaching performance can influence selection and preparation of teacher candidates. Regardless of how one views the flow of influence in the full model, Step 2 is the linchpin in the process. Effective teaching behavior influences or adds value to PK-12 student learning. Effective teacher education yields teachers who behave in ways that influence or add value to student learning.

To acquire useful information on teaching behavior at our own institution, we have relied primarily on two classroom observation systems: The Teaching Performance Record (TPR) and the Classroom Assessment Scoring System (CLASS). CLASS is a set of 10 observation rating scales that have been used in classrooms nationally (Pianta, La Paro, Payne, Cox, & Bradley. 2002). The CLASS captures data on the social, instructional, and managerial elements of teachers' interactions with their students. It is designed to address questions of educational and teaching quality. CLASS provides a common metric, vocabulary, and descriptive base for teaching and classroom processes, primarily in pre-K through grade 5. The intent is to measure variation in both instructional and social dimensions of the classroom environment through observation. In so doing, information from the CLASS provides a mechanism by which classroom practices can be gauged and improved.

The CLASS was developed based on the scales used in multiple large-scale classroom observation studies including the NICHD Study of Early Child Care. There are four major components of the CLASS: emotional climate, management, instructional support, and instruction in content areas. Within each of these components, there are three or four seven-point scales used to assess various dimensions of behavior.

The Teaching Performance Record (TPR) is a system that has observers record "signs" of behaviors within defined time periods (University of Virginia, 2005; McNergney, 2006). Observers record the presence or absence of some 105 items of teaching behavior; they do not judge or rate their observations. The records are later scored in various ways. For instance, patterns of behavior can be identified that represent a teacher's attention to constructs that are relevant to PK-12 student learning. These include Opportunity to Learn, Goal Setting, Scaffolding Pupil Engagement, Achievement Expectations, Application and Practice, and the like (Brophy & Good, 1986; Brophy, 1999; McNergney, 1988). The records can also be scored by reorganizing teaching behaviors according to programmatic goals and state teaching standards required for certification.

The TPR also offers opportunities to collect data on the PK-12 students in the classroom. Observers note signs of student involvement in lessons and their attention to classroom rules of behavior.

The TPR has laptop, hand-held, and paper-and-pencil versions. TPR training involves the use of a video database containing examples of teaching behaviors of interest as they are demonstrated in various classroom contexts—that is, in classrooms that vary by grade, academic level, subject area, number of students, and cognitive demands of the lesson.

Both CLASS and TPR have growing bodies of validity evidence. Clinical supervisors use both instruments to provide feedback on teaching performance in clinical settings and online. The instruments are being used at various points in the pre-service program to improve teaching practice.

Is It Feasible to Conduct Teacher Education Program Evaluations Within and Across Settings?

The feasibility of teacher education program evaluation depends on a number of factors. Within-group, within-program, or within-setting program evaluations require the necessary data. Evaluations across settings, programs, or groups require multiple sites with comparable data. An evaluation plan must be realistic, prudent, and implemented diplomatically if it is to succeed. This means placing a premium on practicality and minimizing disruption in on-going programmatic activities. The evaluation needs to produce information from various sources and of sufficient value to justify the resources expended—cooperation and frugality count.

One way we have begun to address the feasibility issue with regard to the collection of university student data is via the establishment of a research pool similar to that long used in psychology departments. Our School of Education instituted a student research requirement and formed a research pool of student participants. The requirement stipulates that newly enrolled teacher education students either participate in five hours of research studies or complete substitute assignments each year for the life of their programs.

TNE, in cooperation with the office of teacher education, creates opportunities for students to participate in research studies designed to address various aspects of the model in exchange for credit toward their research requirement. Of the 165 students entering the education school the first year of the requirement, 153 or 93 percent joined the pool. The remaining students will pursue research studies independently, or they will complete five hours worth of equivalent work. Year Two had a 95 percent participation rate.

To participate in the pool, students must consent to the use of specific core data on themselves and participate in a set of common studies. The core data includes records of high school and college courses taken, grades received, and SAT, PRAXIS, and GRE test results. Pool participants also agree to complete a collection of common studies intended to capture a range of information on each student. Common studies include: (1) an In-Take Survey designed to gather data on students' attitudes, abilities, and beliefs about education; (2) The Interactive Teaching Assessment that simulates real-time interactive teaching in a middle-school classroom to gauge students' pedagogical content knowledge; (3) a Q-Sort Exercise that uses a card-sort activity to measure the qualities, attributes, and beliefs of teacher candidates; and (4) the Personal Experiences Inventory that measures various aspects of students' lives that may influence their future as teachers. These measures of characteristics, in turn, can be examined in relation to teaching behavior (as measured by CLASS and TPR).

Can Teacher Education Research and Evaluation Operate Within the Bounds of Propriety?

What is proper, right, and ethical in the conduct of teacher education research and evaluation is everyone's concern, but formal university attention to this issue falls under the purview of the Institutional Review Board (IRB). The IRB sanctions and monitors all research performed under the aegis of TNE, as it would any research in any college or university setting. The board is comprised of 17 faculty members from across the university. The primary function of the IRB is to ensure compliance with federal and university regulations for non-medical, behavioral research involving human participants. The IRB standard operating procedures regarding the review of human participant research assure that the risks of such research are compatible with the expected benefits, and that participants always have a free and fully informed choice before giving their consent to participate.

One of the first issues on which we consulted the IRB was the formation of the education school participant pool. Our communications revolved around the board's concerns about the possible threat of coercion of university students. To address IRB concerns, we developed a set of operating procedures that follow as such: (1) The idea of the pool was to be introduced to the students by a faculty member from the College of Arts and Sciences and the actual five-hour research expectation was to become a program requirement established by the Office of Teacher Education; (2) Students who did not wish to participate in the pool had to be given alternative assignments that would allow them to meet the education school's new research requirement; (3) For every pool study, whether administered on-line or face-to-face, the investigator would have to seek and receive IRB approval; (4) Once concluded, all investigators had to offer a debriefing for participants; and lastly, (5) Every study had to be of educational value for its participants.

We also sought approval from the IRB for the use of the TPR and CLASS to be used with all student teachers in order to track their teaching behavior over time. Successful implementation of this plan required approval of both the Institutional Review Board and the teacher education faculty. The IRB was concerned primarily with the quality of supervision the student teachers would receive; that is, the board wanted to ensure that the normal duties of supervisors to provide relevant feedback to students would not be disrupted by their efforts to collect data that would be used for research purposes. The possible threat to the quality of supervision issue was mitigated by faculty acceptance of the systems as natural parts of their programs. In other words, supervisors would begin to use these systems as a natural consequence of their jobs, providing feedback based on results from the use of the systems. To learn the systems, all supervisors, who are also doctoral students, now take a course on evaluation of teaching. The course offers training in the use of the CLASS and the TPR, as well as other methods of observation and feedback. We have taken one step toward making systematic observation with common elements across programs becoming the rule, not the exception.

DISCUSSION

As Cochran-Smith (2006) has observed, there are many positive aspects to the emerging evidence focus of teacher education. The movement holds the potential to transform both research and practice in the field. The concomitant danger is that, if researchers and policy makers take too narrow a perspective on what constitutes evidence, progress will be threatened. The federal government's reliance on randomized clinical trials as the pre-eminent strategy for investigating teaching and teacher education drives Cocharn-Smith's concern. Others worry about the potential of policy makers to misuse William Sanders's (1998) work on value-added modeling (VAM). "Misuse" has been characterized as over-reliance on PK-12 student academic achievement as the metric of teacher education success, while simultaneously deemphasizing the importance of teacher preparation (Lasley, Siedentop, & Yinger, 2006).

Nonetheless, in the end, teacher education must be and will be defined at least in part in terms of PK-12 student learning. Student learning will "pull" the system to attend to children's needs. Knowledge of student learning may be expected to guide how we select for and develop pre-service teachers' characteristics, how we encourage teachers to plan, teach, and evaluate, how we use PK-12 students' thinking about facing academic challenges, and ultimately, how teacher educators judge their efforts. But student learning can and must be expected to exert reciprocal influence in order to improve teacher education practice. Knowledge of student learning must "push" the system back the other way, driving teachers to think about their actions in terms of the concomitant influences on students and shaping programs in terms of the behaviors of their students. Like other professions, teacher education will train to achieve particular ends and use knowledge of results to shape training.

Skeptics will be convinced of the value of teacher education when researchers and evaluators produce evidence of effectiveness. Such evidence will come in a variety of forms at a variety of points during and after training. To obtain evidence will require careful attention to the accuracy, utility, feasibility, and propriety of research and evaluation. Investigators must ask the right questions in order to be able to offer directions for shaping program processes that enhance participants' teaching abilities. Investigators need to concentrate not only on the desirability of evidence but on the likelihood that it can be collected and interpreted within budget and on time. And all those involved in examining teacher education programs will be duty bound to protect the integrity of their investigations and the rights of those involved, or risk debasing the entire process. As we address these challenges in our own communities, we will build models of professional practice that will help us judge our own and each other's efforts.

This manuscript was made possible in part by a grant from Carnegie Corporation of New York, the Ford Foundation, and the Annenberg Foundation. The statements and views expressed are solely the responsibility of the authors. An earlier version was presented at the annual meeting of the American Educational Research Association, San Francisco, CA, 2006.

Notes

1. Bank Street College of Education, Boston College, California State University, Northridge, Florida A&M University, Michigan State University, Stanford University, University of Connecticut, University of Texas at El Paso, University of Virginia, University of Washington, University of Wisconsin, Milwaukee.

References

Bredo, E. (in press). Conceptual confusion and educational psychology. In P. Winne, P. Alexander, & G. Phye (Eds.). Handbook of educational psychology. Fairfax, VA: Techbooks.

Brophy, J. (1999). Teaching (Educational Practices Series No. 1). Geneva: International Bureau of Education [On-line]. Available: http://www.ibe.unesco.org/International/Publications/EducationalPractices/prachome.htm/

Brophy, J. & Good, T.L. (1986). Teacher behavior and student achievement. In M. Wittrock (Ed.), Handbook of research on teaching (pp. 340-370). New York: Macmillan.

Cochran-Smith, M. (2006, January/February). Taking stock in 2006: Evidence, evidence everywhere. Journal of Teacher Education, 57(1), 6-12.

Cochran-Smith, M., & Zeichner, K.M. (Eds.) (2005). Studying teacher education: The report of the AERA Panel on Research and Teacher Education. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

Dunkin, M.J. & Biddle, B.J. (1974). The study of teaching. New York: Holt, Rinehart and Winston, Inc.

Erickson, F. (2005). Arts, humanities and sciences in educational research and social engineering in federal education policy. Teachers College Record, 107, 4-9.

Joint Committee on Standards for Educational Evaluations (1981). Standards for evaluations of educational programs, projects, and materials. New York: McGraw-Hill Book Company.

Lasley, T., Sideentop, D., & Yinger, R. (2006). A systemic approach to enhancing teacher quality: The Ohio model. Journal of Teacher Education, 57(1), 13-21.

McNergney, R.F. (Ed.) (1988). Guide to classroom teaching. Boston: Allyn & Bacon.

McNergney, R.F. (2006, April 8). Judging the value of teacher education. Paper presented at the annual meeting of the American Educational Association, San Francisco.

McNergney, R.F., Cohen, S, Hallahan, D., Kneedler, R. & Luftig, V. (2002). The teaching assessment initiative of the teachers for a new era. Unpublished manuscript, University of Virginia, Charlottesville, VA.

Mitzel, H.E. (1960). Teacher effectiveness. In C.W. Harris (Ed.), Encyclopedia of educational research (3rd ed.) (pp.1481-1486). New York: Macmillan.

Paige, R. (2002). Meeting the highly qualified teachers challenge: The secretary's annual report on teacher quality. Washington, DC: U.S. Department of Education, Office of Postsecondary Education.

Pianta, R. C., La Paro, K. M., Payne, C., Cox, M. J., & Bradley, R. (2002). The relation of kindergarten classroom environment to teacher, family, and school characteristics and child outcomes. The Elementary School Journal, 102(3), 225-238.

Pianta, R. C. (2003, March). Professional development and observations of classroom process. Paper presented at the SEED Symposium on Early Childhood Professional Development.

Washington, DC.

Sanders, W.L. (1998). Value-added assessment. The School Administrator, 55(11), 24-27.

Shulman, L.S. (1986). Paradigms and research programs in the study of teaching: A contemporary perspective. In M.C. Wittrock (Ed.) Handbook of research on teaching (3rd ed.), (pp. 3-36). New York: Macmillan Publishing Company.

University of Virginia (2005). Teaching Performance Record. Charlottesville, VA: University of Virginia.

  Cite This Article as: Teachers College Record, Date Published: October 26, 2006

http://www.tcrecord.org ID Number: 12814, Date Accessed: 12/5/2006 1:28:45 AM

by Jian Wang, Michael Strong & Sandra Odell — 2004

Mentor-novice collaborative reflection about teaching is crucial to the development of novices' professional knowledge. However, few studies examine content and forms of mentor-novice conversations and opportunities that such interactions create for developing professional knowledge. Drawing on observation data from two U.S. and two Chinese mentor-novice pairs in induction contexts, this study analyzed the content and forms of mentor-novice conversations about novices' lessons. We found that the U.S. and Chinese mentor-novice interactions were different in focus and form, and these differences were likely related to the curriculum structures and organization of teaching and mentoring in each country. The interactions either offered or restricted novices' opportunities for developing professional knowledge necessary for reform-minded teaching.

 Mentor-novice conversations about teaching are important to the development of teachers' professional knowledge (Wang & Odell, 2002) and, thus, to the improvement of teaching practice in ways consistent with the reform-minded curriculum standards (Austin & Fraser-Abder, 1995). Drawing on mentor-novice conversations about novices' lessons from two U.S. and two Chinese elementary mentor-novice pairs in induction contexts, we argue that the foci and forms of mentor-novice conversational interactions reflect the curriculum structure and teaching and mentoring organizations in which mentors and novices work. These interactions seem to offer different opportunities for novices as they learn to teach. We argue that allowing mentors and novices opportunities to interact about teaching may not help novices learn to teach in reform-minded ways.

TEACHERS' PROFESSIONAL KNOWLEDGE AND MENTOR-NOVICE INTERACTION

Over the past decade, various professional organizations have established curriculum and teaching standards to help transform teaching practice and improve the quality of student learning (National Council for the Social Studies, 1994; National Council of Teachers of English and the International Reading Association, 1996; National Council of Teachers of Mathematics, 2000; National Research Council, 1996). These standards are being incorporated into curriculum and teaching requirements at the state and school levels in spite of the debates about the effects of such approaches on the reform of teaching (Apple, 2001; Berliner & Biddle, 1996; Cochran-Smith & Fries, 2001).

To teach according to the new standards, teachers are asked to develop knowledge and teach in ways that help children acquire knowledge, skills, and dispositions for their future. In particular, teachers need to understand the subject matter that they are required to teach (Ball & Bass, 2001; Ball & McDiarmid, 1989) and develop flexible representations of subject matter to various groups of students (Grossman, 1990; Shulman, 1987; Wilson & Berne, 1999). They need to understand how diverse children at different developmental and intellectual levels learn, and what influences their learning (Grimmett & MacKinnon, 1992; Ladson-Billings, 1999). They need to analyze their teaching, argue the alternatives and apply their thoughts to uncertain and irregular contexts (Floden, Klinzing, Lampert, & Clark, 1990; Kennedy, 1991; Lampert & Clark, 1990; Leinhardt & Greeno, 1986). These assumptions about teacher knowledge have provided a framework for teacher education and professional development standards (Interstate New Teachers Assessment and Support Teaching Consortium, 1992; National Council for Accreditation of Teacher Education, 1998, 1999; Odell & Huling, 2000) and are helping shape policy and programs in teacher education and professional development (Cochran-Smith & Fries, 2001; Darling-Hammond & Ball, 1998; Sweeny & DeBolt, 2000).

Teacher education reformers propose teacher mentoring as an important strategy for helping novice teachers develop professional knowledge (Holmes Group, 1986, 1990). Scholars argue that mentor-novice relationships, properly structured, can offer novice teachers opportunities to develop subject-matter understanding (Feiman-Nemser & Parker, 1990). Mentors can help novices learn how to represent subject-matter understanding in their teaching (Huling-Austin, 1992; Wang & Odell, 2002). Mentors can support novice teachers in connecting knowledge of teaching to different kinds of students (Kennedy, 1991), help novices learn how to analyze teaching and transform practices (Cochran-Smith, 1991), and to teach in ways consistent with reform-minded curriculum and teaching standards (Austin & Fraser-Abder, 1995).

It is assumed that the success of teacher mentoring in helping novices develop professional knowledge resides in collaborative and reflective interactions between mentor and novice about teaching (Beasley, Corbin, Feiman-Nemser, & Shank, 1996; Cochran-Smith & Lytle, 1999; Feiman- Nemser, 2001). Such an assumption is consistent with a sociocultural perspective about learning. A focus on mentor or novice teaching reflects the idea that all knowledge is situated in and grows out of a context (Brown, Collins, & Duguid, 1989; Rogoff, 1984). The unique relationship between mentor and novice offers opportunities for a novice to access and internalize higher-order-social functions that he or she did not possess in the first place and gradually become part of the experienced community (Bakhtin, 1986; Lave & Wenger, 1991; Vygotsky, 1994). Interactions between mentor and novice offer an opportunity for the experienced to identify the zone of proximal development of the inexperienced and provide the necessary scaffolding to help the inexperienced move to the level beyond their own performance in learning (Tharp, 1988; Vygotsky, 1978).

Two lines of argument and relevant research support or challenge the assumption that mentor-novice conversations about one another's instruction are an effective way to support teachers in developing the professional knowledge necessary for effective teaching. The first argument suggests that lesson-based teacher interaction is important in helping teachers develop professional knowledge and transform existing teaching practice (Hiebert, Gallimore, & Stigler, 2002). Such an argument finds its support in several lines of research on teacher learning and teachers' knowledge. Research on the differences between expert and inexperienced teachers suggests that teachers' knowledge is event-structured and context-based and practical in nature (Carter, 1990; Elbaz, 1983) and that learning to teach should be situated in the context of teaching (Cochran-Smith & Lytle, 1999; Richardson, 1996). Chinese mathematics teachers are found to have a deeper understanding of mathematics concepts and flexible representations of these concepts in teaching (Ma, 1999), and their observations and discussion of each other's teaching are seen to contribute significantly to their knowledge (Paine, 2001; Wang & Paine, 2003). Lesson-based interactions among Japanese teachers are found to be important in helping them develop effective teaching practices as envisioned by the U.S. reform standards (Hiebert & Stigler, 2000; Lewis, 2000; Lewis & Tsuchida, 1998; Stigler & Hiebert, 1999).

The second argument casts doubt on the function of lesson-based mentor-novice conversations on novices' professional knowledge and, thus, on novices' learning to teach in reform-minded ways. Several research studies support this position. One comparative study showed that mentor teachers from different countries held different beliefs about teaching and mentoring associated with the nature of the curriculum system and teaching organization in which they worked (Wang, 2001). Work by Little (1990a, 1990b) showed that teacher mentoring and discussion about teaching, structured in an individualist culture of teaching, does not help form effective and collaborative teacher relationships that are crucial for developing professional knowledge. A mentoring relationship that follows a situated perspective of learning could become a conservative force that helps reproduce the existing culture and practice of teaching instead of transforming it (Cochran-Smith & Lytle, 1999; Cochran-Smith & Paris, 1995).

In spite of being research-based, neither argument has sufficient evidence to present a solid challenge. The first argument, although supported by studies of interactions among groups of teachers, lacks support from research that directly analyzes mentor-novice conversations and their consequences. The second argument relies heavily on interview and survey research from settings where mentor teachers work in traditional contexts, and little attention is paid to how mentors and novices talk about their teaching and what topics they discuss.

Thus, the question arises as to how the foci and forms of mentor-novice interactions about teaching vary from one context to another, and what the consequences of different interactions might be. The exploration of this question is central to having teacher mentoring research move from examining program descriptions (Feiman-Nemser & Parker, 1992; Klug & Salzman, 1991; Odell & Ferraro, 1992; Strong & St. John, 2001), to examining the mentoring of individual mentor-novice pairs (Achinstein & Villar, 2002; Strong & Baron, in press; Wang & Paine, 2002). Comparative research on the foci and forms of mentor-novice discussions in various contexts of teaching and mentoring may be especially valuable in sharpening our understanding about the impact of mentor-novice interactions and discern unique and familiar interactions as well as contextual and universal interactions.

Drawing on observational data from two U.S. and two Chinese mentor-novice pairs, the present study explores three questions that directly address these issues: (1) How do the form and content of mentor-novice conversations about novice teaching compare and contrast in the two settings? (2) What opportunities do the mentor-novice conversations in each setting offer novice teachers to help them develop knowledge necessary for reform-minded teaching? (3) How do the mentor-novice conversations reflect the broad context of curriculum and the organization of teaching and mentoring in each country?

DATA AND METHOD

PARTICIPANTS

The participants were two U.S. mentor-novice pairs: Tanya1 and Kevin, and Peggy and Elaine, and two Chinese mentor-novice pairs: Cao and Xue, and Liu and Sun. We chose these pairs because their similarities and differences offered us the opportunity to examine the study's questions, particularly as they relate to cross-cultural comparisons of mentor-novice conversations.

All the novices were first-year teachers working in public elementary schools. The two U.S. novices, Kevin and Elaine, taught fifth grade in different schools. Kevin taught 24 ethnically diverse students, and Elaine taught 30 mostly Caucasian students in a mixed suburban rural community in California's central coast. Kevin graduated from an undergraduate teacher education program in a neighboring state, while Elaine held a master's degree in language arts and went through a post-BA teacher education program in the state where she was teaching. In contrast, the two Chinese novices, Xue and Sun, had similar education backgrounds, graduated from the same normal school,2 and taught in the same school in the urban area of Shanghai, China with 50 students from similar cultural backgrounds. However, Xue taught second grade, whereas Sun was a first grade teacher.

The novice teachers taught under very different curriculum contexts in each country. In the U.S. setting, the two novices were urged to teach according to national and state curriculum standards. However, such curriculum standards were not always consistent with the textbooks and other school curriculum materials. Standards were not strictly implemented with regard to topic requirements, pedagogical suggestions, or schedule (Cohen & Spillane, 1992). The two Chinese novices taught with textbook and teachers' manuals that were consistent with the national centralized curriculum. This curriculum was strictly implemented and teachers had no choice but to follow it (Wang, 2001).

The novice teachers had different teaching responsibilities and were organized differently for their instruction in each country. The two U.S. novice teachers, like most U.S. elementary teachers, were generalists who taught all subjects. They stayed with their class throughout the day, reflecting an individualist organization of teaching (Feiman-Nemser & Floden, 1986). The two Chinese novices, like most Chinese elementary teachers, were subject specialists. Xue taught language arts while Sun was a mathematics teacher.3 They taught two lessons in two different classrooms each day and stayed in their teaching-research group office after their instruction. The teaching-research group frequently brought teachers together who taught the same subjects to plan, observe, and reflect on each other's teaching. Thus, those teachers worked in a contrived teaching organization (Wang & Paine, 2003).

All the mentors in this study had more than 15 years elementary teaching experience before becoming mentors. However, the two U.S. mentors worked full time, mentoring 12 to 15 novices at various grade levels in different elementary schools. They were required to meet with their novices weekly, and to observe teaching and provide feedback at regular intervals. The two Chinese mentors, Cao and Liu, were experienced teachers teaching the same subject at the same grade level in the same school as their novices. Their mentoring work was structured around frequent public lessons that all novice teachers were required to teach so their colleagues could observe and analyze, using the national curriculum framework for their subject as a basis. The two Chinese mentors frequently observed novice teachers' lessons and provided feedback to novices after their observation.

The two U.S. mentors worked in a program where the goal was to help new teachers learn to teach according to national and state professional standards for the teaching profession. In support of this goal, the program offered a professional development seminar to mentors once a week. It also provided many training sessions as well as mentoring resources, tools, and protocols. The two Chinese mentors worked in a school-based induction program. Although the school pushed teachers to teach by focusing on students' interests and conceptual understanding, the mentoring program was developed to help novices move into the existing culture of teaching.

DATA SOURCES

The data included videotaped novice lessons and either audio- or videotaped conversations between mentor and novice teachers from the four mentor-novice pairs. These data originated from two research projects. The U.S. data were part of the study, The Effects of Mentoring on New Teacher Development and Student Outcomes,4 conducted at the New Teacher Center, University of California, Santa Cruz. The Chinese data originated from a cross-national research project, Learning from Mentors,5 at the National Center for Research on Teacher Learning (NCRTL), Michigan State University. The two U.S. pairs were randomly selected from 20 such cases, all of which were reviewed first to determine that they matched the Chinese data sets that included similar lesson- observation and pre- and postlesson discussions. There was no reason to believe that these two cases were in any relevant manner atypical from the others in the sample. For example, in a related study (Strong & Baron, in press), it was found that all the U.S. mentor-teacher pairs showed similar characteristics with regard to how the mentor made suggestions and with respect to the topics discussed. The two Chinese mentor-novice pairs were chosen for analysis here from seven cases of mentor-novice relationships in the project. They were at the elementary level and had data sets that included post-observation conferences similar to the reflective conferences presented in the U.S. mentoring cases. Three of the other Chinese cases were at the secondary level, and the other two were at the elementary level but had no corresponding data set. Furthermore, the observations analyzed here were made during the earlier part of novices' first year teaching. One difference between the two samples concerns the subject matter being taught: language arts in one Chinese case and math in the other Chinese case, while language arts was the subject matter in both U.S. cases. This was a limitation imposed by the foci of the two original projects and may have implications for the findings. However, in spite of the subject differences between the two Chinese cases, the difference in the findings between the two Chinese cases is considerably smaller than between the Chinese cases and the U.S. cases.

CONVERSATIONS

In the U.S. setting, the mentoring program required mentor teachers to review the teacher's lesson plan, make suggestions based on novices' questions and concerns and the California Standards for the Teaching Profession, and determine a focus for observation before novices' lesson. The postlesson conversation, known as the reflecting conference, enabled mentor and novice to review the lesson, ponder the success of the original plan and any deviations from it, give and respond to feedback, and discuss next steps during mentor's visit. While the mentor would come to these conferences with an agenda of questions to ask and areas to cover, the actual interactions had more the character of a conversation than an interview and ample room for the teachers to discuss whatever was on their minds.

In the Chinese setting, mentor-novice pre- and postlesson conversations were also a requirement from the school-based mentoring program. However, the program did not impose the agenda for the conversation. For both cases, several prelesson conversations could occur before the novice taught and includes mentor-novice coplanning the novice's lesson, mentor reviewing the novices' lesson plan, and/or the novice teaching the lesson to the mentor. In each case, the mentor would critique and make suggestions based on the novice's needs at the moment and in relation to the requirements of centralized curriculum materials. The postlesson conversations were often opportunities for the mentor to review and critique the novice's teaching, identify places where the novice teacher needed to pay attention, and make suggestions for improving teaching. Conversations were often infused into discussions that occurred in a teaching research group where other teachers in the same school who taught the same subject came together to discuss teaching.

In this study, one novice lesson and the post-lesson conversations between mentor and novice from each of four cases were analyzed. The tape-recorded prelesson conversations between the Chinese mentor and novice were unavailable and prevented us from including the prelesson conversations in our current analysis.

DATA CODING

The four videotaped lessons, on which mentor-novice conversations were based, were analyzed in three steps as background to the conversations themselves. The first author transcribed each lesson and translated the Chinese lessons. Then the activity chunks in each lesson were coded and a summary description of each lesson was developed with a focus on what had taken place in each part of the lesson.

The four audio- or videotaped mentor-novice conferences were transcribed and coded for content and form using the following specific procedures. First, each conference was transcribed (the Chinese conferences were translated) and then coded for initiation and response sequences. The major issues of discussion were identified and specific topics were noted as they emerged in the discussions, as suggested by Strauss and Corbin (1990).

Second, each of the emergent topics identified from the above procedure was labeled, as referring either to (1) teaching, (2) subject matter, (3) students, or a combination of these three. The few topics that fell outside these three categories were categorized as ‘‘other.'' The category ‘‘teaching'' describes conversation about what and how the novice or mentor in a general sense did a particular step of the lesson, teaching techniques, approaches, or instructional tips. ‘‘Subject matter'' refers to discussion about the subject concepts and content taught in the lesson including its meanings, understanding, and sources. By ‘‘students,'' we refer to discussion about children in the lesson or in general. This may be about their behavior, their ability, their learning styles, or their response in a given situation. When a topic addressed any two of the categories, it was labeled as a multifocused topic.

Third, each conversational topic was then coded for speech acts, specifically illocutionary acts (Austin, 1962), indicating any kind of intention found in the samples, which led to seven categories of intentional speech acts: (1) compliment, (2) critique, (3) question, (4) agree, (5) disagree, (6) explain, and (7) describe. Then each topic in a mentor-novice conversation was recoded using these seven categories in the same manner we coded the focus of mentor-novice interactions.

Fourth, each conversational topic was coded for the degree of specificity of the speech acts. Four categories emerged from this coding: (1) unelaborated topics that were presented without any example or reason as a support, (2) example-based topics where example was used to support the idea presented, (3) reason-based topics that were presented with reasons but without examples, or (4) reason- and example-based topics. Then each topic in a mentor-novice conversation was recoded using the same four specificity categories in the same manner that mentor-novice interactions were coded.

All of the topic coding was checked for concurrence by at least two researchers. Any differences were discussed among authors until agreement was reached.

DATA ANALYSIS AND ITS LIMITATIONS

The first part of the analysis addresses the research question: How do the form and content of mentor-novice conversations about novice teaching compare and contrast in the two settings? Similarities and differences between the two cases within and between countries were assessed. These findings were then related to the literature on what novice teachers need to learn in order to teach in reform-minded ways to address the second research question: What opportunities do the mentor-novice conversations in each setting offer novice teachers to help them develop knowledge necessary for reform-minded teaching? The findings were then related to the literature on the context of teaching and mentoring to address the third research question: How do the mentor-novice conversations reflect the broad context of curriculum and the organization of teaching and mentoring in each country?

This study has three obvious limitations. First, since we did not analyze the novices' subsequent teaching practice, we were not able to identify the consequences of mentor-novice interaction on what novices actually did, although learning opportunities offered in these interactions are discussed in relation to relevant literature. Second, this study only had available for analysis a set of rather formal conversations related to lesson planning and reflection. We recognize that other more spontaneous conversations may have been characterized by different content and linguistic forms, but such conversations were not recorded and hence not available for analysis.

U.S. MENTOR-NOVICE RELATIONSHIPS

THE CASE OF KEVIN AND HIS MENTOR, TANYA

Kevin's lesson lasted about 55 minutes and focused on helping students identify and use transition words in their writing. His mentor, Tanya, observed the lesson. Kevin's fifth-grade class had 24 students from diverse backgrounds. They sat in pairs in four rows facing the board. Kevin started the lesson asking students to explain the meaning of transitions. Some students offered examples, such as: changing from playing a game to playing basketball; or, tadpoles could change to frogs. Kevin then told the class that the topic for the lesson was how to use transition words in writing to show idea changes.

Next, Kevin asked a student to get up, go to the pencil sharpener, and then sharpen her pencil. He then asked the class to identify the words in his directions that indicated idea changes. Of four students called on, two were able to identify ‘‘then'' and ‘‘first'' correctly, but the other two gave improper answers by pointing out words like ‘‘get up'' and ‘‘go'' as transition words. Kevin praised students who gave the right answers and told the other two that their answers were not correct. He offered no further explanation. Kevin then gave students a copy of a story and asked them to identify the transition words as he read aloud: ‘‘Molly ran into the house as soon as she heard thunder and she immediately hid under the covers. Afterwards, Molly climbed out of the bed since the storm was no longer a threat. The rain nevertheless prevented Molly from returning outside to play.'' One girl told the class ‘‘returning'' was a transition word. Kevin read the story again and stopped at each sentence asking students to identify the transition words. Although most students called on were able to find the right words, two students told the class that ‘‘ play'' and ‘‘ returning'' were transition words. Kevin said these answers were incorrect without explanation, and asked the students to circle the correct ones on their copies. He put the transition words that students had so far identified on the board. Again, three students offered incorrect examples: ‘‘go,'' ‘‘do,'' and ‘‘from.'' Kevin responded that these words were indicators of actions rather than transition words.

Kevin then read three news stories about young athletes from a local newspaper and asked the class to identify the transition words. He read through the first news story on a cross-country runner and then reread each sentence stopping for the class to identify transition words. Three students found the correct words while one boy mistakenly regarded the words, ‘‘basketball court,'' as a transition. Kevin recommended that he choose ‘‘which'' instead, using the comma that separated ‘‘which'' from the previous part of the sentence as an indicator of transition. He read the other two news articles in the same manner, and again asked the class to identify transition words. One story was about a young gymnast and the other about a young tennis player. This time only one student mistakenly identified a transition word.

In the next part of his lesson, Kevin asked the students to write a short paragraph about their school or daily life using at least four or five sentences and two transition words that they had learned in the lesson. He had students brainstorm what they could write, gave them more ideas, and finally gave specific directions for the assignment. For the next 20 minutes, students worked independently on their paragraphs while Kevin walked around and helped individuals.

Kevin ended the lesson by reading four students' compositions and asking the class to identify transition words. He read the first two stories sentence by sentence and one student proposed ‘‘basketball game'' as a transition word, although other transition words were correctly identified. For the next two stories, Kevin raised his voice at each transition word.

Kevin and Tanya's post-lesson conference followed Kevin's 55-minute lesson. Kevin and Tanya's conversation contained 20 topical units related to teaching, students, or both in the order shown in Table 1.

Of these units, nine were related exclusively to teaching (1, 2, 9, 10, 11, 13, 15, 18, and 20) with (9) and (15), as well as (10) and (13) focusing on the same teaching skill. Six units concerned students (3, 4, 6, 7, 14, and 16) with (6) and (14) dealing with the same issue about transition words that students had identified in the lesson. Three units (8, 12, and 17) were related to both teaching and students while the remaining three (5, 19, and 21) focused on the mentor-novice discussion with (19) and (21) on the same issue about how the novice felt about conversing with the mentor.

The specific topics covered in each unit reflected the same general pattern, with a dominant focus on teaching and students, particularly individual students. Subject-matter content or students' understanding of it received little attention, a surprising result since some students failed to grasp the concept of transition words and Kevin did not explain their misunderstandings. As shown in Figure 1, 43% of the conversational topics were related to students and 48% to teaching with only 5% related to subject matter. In the following example, Kevin shows how he was focused on student behavior when walking around the classroom rather than on the understanding of transition words:

What I was seeing was that most of them had a good start to things, and they were taking a chance to pause. So, I was kind of doing the proximity thing. Going around to the ones I know I should be talking with. I myself, I can't keep them all writing at the same time, but I

Table 1: Units of conversations in each mentor-novice conference

U.S. Tanya and Kevin U.S. Peggy and Elaine Chinese Cao and Xue Chinese Liu and Sun

1. Novice's feeling of the lesson

2. Novice's goals of the lesson

3. Individual student work

4. Discuss student background

5. Mentor's observation notes

6. Students' transition words

7. Students' attention span

8. Help ELL students

9. How to introduce assignment

10. How to read sample articles

11. Give assignment requirement

12. Encourage students to speak

13. How to read sample articles

14. Students' transition words

15. How to introduce assignment

16. Process of student writing

17. How to share student work

18. How to close the lesson

19. Mentor-novice discussion

20. How to follow up the lesson

21. Mentor-novice discussion

1. Novice' feeling about the lesson

2. Novice planned activities

3. Students ideas in writing

4. Individual student work

5. Class environment and student learning

6. Students' comfortable level

7. Individual student behaviors

8. Expectations for students' learning

9. Individual student behaviors

10. Reinforce student behaviors

11. Aspects need to be improved 1.

Mentor's feeling about the lesson

2. Meanings of several words taught

3. Effects of practice game

4. Use words beyond the text in teaching

5. Effects of practice game

6. Teaching style

7. Structure of the lesson

8. Use of lesson time

9. Coverage of lesson content

10. Applying teaching principle

1. Mentor' s feeling about the lesson

2. Students' thinking about mathematics

3. Use of manipulatives

4. How to represent mathematics concepts

5. Use of manipulatives, use of lesson time

6. Help students find right ways to sit and write

made it around through the most of the entire room, and they all had at least a good start by the time I got to them.

The few multifocused topics were similarly focused on teaching and students with little attention to subject matter or students' understanding of it. Figure 2 shows that about 17% of the total topics in the conversation were multifocused with 11 % focusing on teaching and students. The following is a typical example initiated by Tanya to help Kevin allow more students access to his instruction:

I'm just a little curious, as far as the English language learners (ELLs), I wonder if there was a strategy you could have used to make the leap even more accessible for the ELLs. You said it verbally, but what we know is often kids that aren't auditory learners also need to see it visually.

With regard to the linguistic forms, the following patterns were evident. Both Tanya and Kevin had almost equal chances to talk. As shown in Figure 3, Tanya took part in 51% of the topics in this conversation and Kevin 49%.

Tanya and Kevin's conversation followed a particular sequence. Tanya started with a description or question about an aspect of Kevin's lesson, and Kevin responded with an explanation that was followed by a further question and explanation or by a compliment plus acknowledgment. Among the 21 units of interaction, Tanya initiated 19 and about 17 of them followed the pattern described. The exceptions were two units initiated by

Tanya complimenting and critiquing an aspect of the lesson. The following is a typical unit of interaction between Tanya and Kevin focusing on students' backgrounds:

Tanya: Is this a pretty a good sampling of the class? (Question)

Kevin: This would be pretty much a sampling of the ability levels of my class, minus some of my ELL (English Language Learner) students. And, um Jessica is one of my better writers. She had a kind of short paragraph in hers but it looks good. There is a transition word between each sentence. And here's one of my struggling writers (show the student's assignment), and he did a very nice job. Um, Shelby would be kind of mid range to a little bit higher and she does pretty well with him. And I just said, ‘‘see her.'' I'm good with that. So, that was kind of across the board of what I have now. (Explain)

Tanya: So this is a good cross section? (Question)

Kevin: Yeah, this is a good cross section. (Agree)

Tanya: So, all of them were able to use transitions. And that was like one of the things I know you were looking for in the lesson.

(Compliment)

The analysis of speech acts also showed that the dominant forms in this interaction were Tanya asking questions and describing what she saw and

then Kevin explaining and agreeing with his mentor. There was little room for Tanya to critique and offer suggestions for Kevin's lesson, or for Kevin to ask questions or disagree. Figure 4 indicates that 25% of the speech acts consisted of the mentor asking questions, 25% were the novice explaining, and 18% were the novice agreeing. These three speech acts together amounted to 68% of the total. In addition, about 6% of the mentor's speech acts were describing and a further 7% were complimenting.

Both Tanya and Kevin's speech acts were mostly unelaborated. That is, for the most part they made their comments or questions without articulating reasons or offering examples. Figure 5 shows that 65% of the speech acts were unelaborated, while 21% were accompanied by examples, 11% by a reason, and 3% added both reason and example. The following is a unit of conversation on encouraging students to speak in class showing Tanya and Kevin's tendency towards unelaborated speech acts:

Tanya: You're asking them ‘‘raise your hand if you can.'' So, I know that you were working on that earlier in the year. (Unelaborated)

Kevin: I have used them since the last time and the class was so much better. Of course, I've got no hands. (Unelaborated)

Tanya: So, it's something you really internalized, using this and it helps a lot. (Unelaborated)

Kevin: Even when I start saying, ‘‘if you know,'' they'll hear me finish that sentence. (Unelaborated)

Then their conversation moved to another topic.

THE CASE OF ELAINE AND HER MENTOR, PEGGY

Elaine's lesson with fifth graders focused on poetry writing and lasted about 70 minutes. Her 30 students were mostly Caucasian and sat in rows facing the board. Elaine started the lesson by reminding students of the process they had used for observing and drawing a tree over the previous few days. She told the class that in this lesson they were going to write a cinquain poem about the tree that they had observed and drawn.

Elaine showed three paintings of a tree that she had drawn, and asked the class to identify their differences. Some students reported that the drawings became increasingly colorful and detailed while others thought they became more creative. Elaine told the class that their own drawings had also become more colorful, detailed, creative, and complex over the past few days.

Next, Elaine introduced the poem structure to the class. It included five lines with a one-word title as the first line, two adjectives for the second line, three -ing verbs in the third line, three words describing the title for the fourth line, and a synonym for the title as the last line. During her explanation, Elaine drew students' attention to the meaning of the term ‘‘synonym'' by asking them to identify other words for ‘‘jazz,'' ‘‘dog,'' and ‘‘friends.'' In each case, students found the right synonyms. She then required students to write a cinquain about their tree in four steps: (1) draft a first version; (2) develop a second version using the thesaurus to help replace some of words in the first version; (3) read both versions to a partner and have the partner listen to how the poems sound and check the spelling; and (4) write the final version.

In the next part of the lesson, Elaine created the following poem with the students providing input for each word in each line: ‘‘Tree. Nice, dark, wonderful. Curving, swerving, waving. A moving, developing shrub. Plant!'' She read it aloud to the class and asked students to replace some of the words in the poem. Based on students' suggestions, she revised the poem as follows, ‘‘Tree. Pleasurable, Gloomy. Twisting, flowering, swing. A growing, living plant. Shrub.'' Elaine read the second version and asked students to judge which poem sounded better. They agreed that the first version was better, but Elaine suggested that they change one word in the first poem. The students proposed that the word ‘‘dark,'' in the first poem be replaced by the word ‘‘gloomy'' in the second.

After reiterating the steps on how to write a cinquain, Elaine asked her students to write their own poems. She walked around the classroom and helped students who had problems writing. During this period, Elaine interrupted students' independent work once to read a student's poem aloud, complimenting the student on his word choice, and reminding the class to read their poems to a partner and check each other's spelling and how the poems sounded.

About 20 minutes later, Elaine read several more poems to the class She read a student's poem and praised her use of the word, ‘‘inviting.'' She then read one of her own poems about the tree in her parents' backyard, stating that it reflected her own feeling about it at the time. Then Elaine asked three students to read their poems to the class and complimented them on some of the words they used. The students then continued writing poems. She assisted those who needed help, while keeping the rest on task.

In the last part of her lesson, Elaine read three more student-composed poems aloud, asking the writers to hold up their tree pictures. The lesson finished with 10 more minutes of student independent writing.

Elaine and Peggy's post-lesson conference lasted about 11 minutes. They covered 11 topic units that were either teaching-focused, student-focused, or both (see Table 1). Units (1), (2), and (11) concerned teaching. Units (3), (4), (6), (7), and (9) focused on student issues, with units (7) and (9) dealing specifically with student behavior. The other three units (5, 8, and 10) were related to both teaching and students.

Looking at specific topics in the conversation, we found that both Peggy and Elaine paid most attention to issues of teaching and students, especially individual students and their behaviors. Subject matter received relatively little attention. As shown in Figure 1, about 59% of the topics in their conversation were exclusively related to students, and 32% were about teaching, while only 5% were concerned with subject matter. Following is a unit of conversation on student comfort level where Elaine and Peggy paid substantial attention to individual students:

Peggy: How do you tell if it's comfortable for the kids? How do you tell if it's working?

Elaine: You judge from their questions, look at their work, see how they are using their materials, listen how they're talking to each other. Um, those kinds of assessments.

Peggy: Okay. Any big successes as you were wandering around or when you were giving instructions that you knew this was working?

Elaine: Well, one, one is Shane who half way through was super frustrated and had crumpled up his poem and concerned with other people's view of his poetry. And so, we got him going again. And he was able to really embody his ‘‘tree'' in his words, and it was absolutely wonderful. And I took that opportunity to read it to the class, and then Austin who's been a struggle couldn't find his tree. So, I gave him an extra. And I said I know that you don't, may not have an emotional attachment to this, but we're still going to look for these elements. And he was able to do it and able to talk to his neighbor about it and come up with some ideas. So, that was absolutely wonderful.

For the most part, topics in this conversation had a single focus. The few multifocused topics covered both teaching and students. As shown in Figure 2, 87% of their topics had a single focus, and 13% were multifocused. The following was a topic in which Peggy complimented Elaine's teaching method and its effects on students' engagement during the lesson:

Your structure of support, whether it be review in the beginning, you do the bouncing ball that we talked about before where you talked a little and you throw it out to the kids. I noticed anywhere around eight to twelve hands being raised. So, you do the bouncing ball to keep that engagement piece going.

With regard to conversational form, Elaine and Peggy had almost equal opportunities to talk with Peggy developing 54% of the topics and Elaine 46%. Their conversation often started with Peggy asking a question or complimenting Elaine on her lesson. An explanation from Elaine, and either further questions from Peggy with explanations, or a compliment plus agreement typically followed this. Nine of the 11 units of their conversation followed this pattern. The other two units started with Peggy's description of a specific part of Elaine's lesson followed by further explication from Elaine, or her agreement. The following was a typical unit of conversation between Peggy and Elaine on assessing students' learning:

Peggy: How did your lesson meet or not meet expectations that you had of yourself perhaps or of the kids? (Question)

Elaine: I think that this is one of the lessons that for the most part have met the expectations. Again, checking in with kids that usually or sometimes don't even get started on things or may revisit and not be comfortable. Those kids when I checked in with them were getting things done. So, in that sense, it has met expectations. Now I still need to, there's not everybody finished, so I will later today do a final check. Because our goal is, one of our goals is to get them posted tomorrow, so having a finished product by the end of today is what we're looking for. (Explain)

Peggy: Right. Right. (Agree)

Elaine: And from walking around what I could tell was that almost everybody is ready for that. (Agree)

Peggy: Yeah, it appeared that walking around, checking in with the kids, having conversations with the kids definitely is, you know, on-your-feet assessment. (Compliment)

The analysis of illocutionary speech acts in this conversation also suggested that Peggy spent most of her time asking questions, complimenting a specific aspect of Elaine's lesson, or agreeing with what Elaine saw, while Elaine responded with agreements or elaborated responses. The mentor spent little time critiquing or making suggestions. From Figure 4 it can be seen that the mentor's most frequent speech acts were questioning (20%), complimenting (14%), and agreeing (11%). For Elaine, the distribution was explaining (20%), and agreeing (24%). As with Kevin and Tanya, Elaine and Peggy's speech acts were mostly unelaborated. From Figure 5 it can be seen that about 67% of their comments were unelaborated, 11% included an example, 5% added a reason and 4% both a reason and an example. The following unit of conversation on individual students' behavior is typical:

Peggy: Jordan is an interesting and delightful little guy, isn't he? (Unelaborated)

Elaine: I think so. (Unelaborated)

Peggy: Yeah. (Unelaborated)

Elaine: And there are lessons where he may not pass the test, but his performance, his outlet, his (inaudible 3 words), his vivid imagination, with what he's interested in. (Elaborated with example)

Peggy: He's unique. (Unelaborated)

Elaine: And Gabe. He's smart. And Gabe could provide that opportunity that would give him that self-confidence and that he's looking for. (Unelaborated)

Peggy: Right. And he started this morning with his jacket on.

(Unelaborated)

Elaine: Not typical. (Unelaborated)

Peggy: Right. But he's just a very unique child. (Unelaborated)

COMPARISON OF THE TWO U.S. CASES

Conversational Foci

The most striking finding was that, in both cases, the conversations focused mainly on teaching and students, especially individual students (90% in each case, see Figure 1). The primary difference between the two cases was that Peggy and Elaine paid more attention to students (58%) than teaching (31%), while Tanya and Kevin focused a little more on teaching (48%) than on students (43%).

Second, both cases dealt hardly at all with subject matter, even in multifocused topics. In the case of Tanya and Kevin, subject matter was the focus only 3% of the time, while Peggy and Elaine discussed subject matter 6% of the time (see Figure 1). The difference here was that Kevin initiated these topics with his mentor, while both Peggy and Elaine introduced subject matter issues into their conversations.

Conversational Forms

The mentors and novices in both cases had equal opportunities to talk, as seen in Figure 3. Most of the time, their conversations also followed the same pattern. Usually it began with the mentor asking a question, describing an aspect of the novice's lesson or making a compliment. The novice responding to the question, further questions from the mentor, and then further explanation, or perhaps further complimenting with the novice agreeing followed this. Seventeen of 21 units in Tanya and Kevin's conversations and 9 out of 11 units in Peggy and Elaine's conversations followed this pattern.

These conversational patterns generated similar distributions of speech acts among the four participants. As seen in Figure 4, question and compliment were the top two speech acts for both mentors (25% and 7% for Tanya, and 20% and 14% for Peggy). Explain and agree were the two top speech acts for the two novices, (25% and 18% for Kevin, and 20% and 24% for Elaine). Peggy tended to offer more compliments (which Elaine acknowledged) and Tanya tended to ask more questions (which resulted in Kevin offering more explanations). Criticisms and suggestions from the mentors were few (see Figure 4).

In both cases, elaborations, when they existed, tended to be in the form of giving examples rather than offering a reason. As shown in Figure 5, only 11% of elaborations were in the form of reasons in the case of Tanya and Kevin and 5% in the case of Peggy and Kevin.

CHINESE MENTOR-NOVICE RELATIONSHIPS

THE CASE OF XUE AND HER MENTOR, CAO

Xue's lesson focused on the structure and use of new characters. It lasted 40 minutes and was observed by Mentor Cao. With her 50 second graders sitting in pairs in four rows facing the board in the classroom, Xue started her lesson reviewing some of the characters that students had learned the previous day. She showed cards with different Chinese characters, and had students say the character aloud. All the students she called on answered correctly.

After the review, Xue put up a small blackboard on which three left parts of Chinese characters were written. She asked several students to make a character by pairing each left part with a right part. After each student answered correctly, Xue led the class in reading the topic for the day aloud: ‘‘How can we structure a character by pairing a left part with a right part?''

Next, Xue showed a character, ‘‘Zhu'' meaning ‘‘live,'' with two side-by-side parts on a card and asked a student to pronounce it and explain its meaning. She flipped a part of the card to cover the left part of the character with a different left part to make a new character that still sounded like ‘‘Zhu'' but meant ‘‘attention.'' She then asked another student to pronounce and explain the new character. As both students answered correctly, Xue put another small blackboard on the wall with the character, ‘‘Zhu,'' meaning ‘‘attention,'' along with its pronunciation, its definition, and a sentence in which the character was used. She began to teach aspects of the character using the following steps.

First, she led the class in reading the character, its definition, and the sentence involving the character aloud, and then asked a student to lead the class in the same manner. Second, she asked several students to explain the meaning of the phrase, ‘‘pay attention to.'' When a student provided an unexpected explanation, she called another student to tell the class the right answer. Third, Xue asked the class to find the similarities and differences between the character, ‘‘live,'' and the character, ‘‘attention,'' and explain the ways in which they remembered the two characters. A girl told the class these characters were pronounced in the same way and a boy said they had different left parts. In the end, Xue led the class in reading aloud the character, ‘‘attention,'' its definition, and a sentence containing the character. Students then took turns reading the sentence aloud.

In the next part of the lesson, Ms. Xue taught each of the following characters, ‘‘Liang,'' ‘‘Yi,'' ‘‘Cheng,'' and ‘‘Piao,'' respectively meaning, ‘‘traffic,'' ‘‘already,'' ‘‘honest,'' and ‘‘beautiful'' using similar steps with slight variation in the order. During this period, whenever a student made a mistake or had a hard time reading the meaning and the sentence, Xue would directly address the mistake by asking another student to give the right answer, or by giving the student a clue.

She finished her lesson with a practice game in which several students took turns standing in front of the class wearing a hat with a new character that they had just learned. Then the student called on other students in the class to answer the following four questions about the character: What is my pronunciation? What do I mean? How do you remember me? Can you put me in a sentence? As students played the game, Xue reminded students about the differences between these new characters. In the end, she asked the class to write these characters as their homework.

Xue and Cao's postlesson conference lasted about 10 minutes. Ten units of conversation were developed with a strong focus on teaching and subject matter as shown in Table 1. Seven units (1, 3, 5, 6, 7, 8, and 10) focused on issues of teaching with (3) and (5) on the same issues of the practice game and (1) and (7) on lesson structure. Three units (2, 4, and 9) focused on subject matter or both subject matter and students.

Looking at specific topics within the conversation, both Cao and Xue paid substantial and almost equal attention to topics related to teaching, subject matter, and students, although no individual students were singled out in their conversation. As shown in Figure 1, 24% of the topics were related to subject matter, 20% focused on students, and 56%, were related to teaching. In addition, many of their topics were multifocused. Figure 2 shows that of the 41% multifocused topics, 3% addressed teaching and subject matter, 16% teaching and student, 12% subject matter and students, and 9% teaching, subject matter and students. Cao initiated the following topic to address using words beyond the text in teaching:

Another issue in your teaching is that all the characters, words, and sentences that you taught were limited to the text. Our school encourages students to know more than the text. So I say you need to move your students beyond the text based upon what they learned from the text. In this way, your lesson will be much more active and reflect the idea of happy learning.

With regard to conversational form, although both Cao and Xue initiated a substantial number of topics in their conversation, the mentor, Cao, was clearly the one who spoke more. Figure 3 shows that the mentor initiated 66% of the topics and the novice 34%.

The conversation between Cao and Xue included three parts. First, the mentor started the conversation with several compliments on various aspects of Xue's lesson, during which the novice asked a clarifying question. Then Cao initiated five conversational units, each starting with a critique on an aspect of the novice's lesson, followed by Xue's response either in the form of an agreement, disagreement, or question. The mentor's further critique and suggestions and the novice's subsequent agreement followed this. Finally, the novice initiated three units by asking questions with the mentor either making a suggestion or criticism, followed by a repetition of the same sequence. Following is an example of such a mentor-novice interaction:

Xue: In teaching this lesson, I divided the text into four parts. I started to teach my students the first two chunks and then they were going to learn the next two chunks by themselves. How about I divide the text into two lessons with one on content and the other on practice? (Question)

Cao: What if your students are unable to understand that much in one lesson? Some parts of the text have a lot of new characters. I think you still need to consider what the requirements for teaching this text are and who your students are. I think you do not have to be strict with time. If you can finish it, okay. If you can't finish it, you can do it later. (Critique and suggest)

Xue: This text is easy. I think it is important for students to have a general picture of the text first. The new characters in it were often used in daily life, like birds and trees. I think my students can understand them. So, I could cut it into two. (Disagree)

Cao: I think we can study other texts together and see which text can be dealt with in one lesson and which text can be taught in several lessons. (Suggest)

The analysis of the speech acts shows that, for most of time, the mentor was critiquing, complimenting, and offering suggestions, while the novice asked questions, agreed, and disagreed with the mentor's ideas. As Figure 4 indicates, 23% of the speech acts in this interaction were Cao's compliments, 20% were her critiques, and 19% were her suggestions. Together they amounted to 62% of the total speech acts. Sixteen percent of Xue's speech acts were raising questions, 13% were agreements, and 6% were disagreements. Following is an example of how Cao complimented Xue on the structure of her lesson:

Your lesson structure is complete in terms of steps. You start from what you taught before and helped them review it. Then you moved toward new content changing the left part of a character. The content of your lesson started from characters, words, and then to sentences. This is good.

Often (about 25% of the time, see Figure 6) Cao followed her critique with a specific suggestion. Here is an example where Cao offered criticism along with suggestions for Xue to improve her lesson on students' writing words:

Another issue is that the part of lesson for students to write the words was too short. I think you should let them use their pens to write something. You do not have to do it for every word but you should let them to practice writing some of them.

Finally, both mentor and novice usually elaborated their ideas by giving reasons or examples. As shown in Figure 5, 25% of the speech acts were accompanied by both reason and example, 31% with reason only, and 15% with example only. Together this accounted for about 72% of all the speech acts. Here is an example of how Cao critiqued the practice game in Xue's

lesson with both reason and example when Xue claimed her game was able to involve more students in learning:

Yes, it can involve a lot of students. However, you need to pay attention to how your students read each sentence. Some of your students were able to read it well, but some

Yes, it can involve a lot of students. However, you need to pay attention to how your students read each sentence. Some of your students were able to read it well, but some do not (example). You should provide feedback immediately and correct their mistakes. Although the little train activity can efficiently involve your students, you still need to pay attention to their mistakes and help them see it (reason).

1. THE CASE OF SUN AND HER MENTOR, LIU

Sun's lesson was 40 minutes long focusing on addition and subtraction involving zero. Her mentor, Liu, observed. The class had 50 first-graders sitting in pairs at desks arranged in four rows facing a small teaching platform at the front of the room. After having students practice several subtraction problems with flash cards, Sun showed a self-made piece of cardboard with a drawing of a birdcage and three birds attached to it. She took three birds from the cage, put them elsewhere on the blackboard, and asked two students to use mathematical sentences to represent what had happened. Both students came up with the answer: 3 - 3 = 0. Sun then showed a picture of two plates, one with four pears and one empty, and asked students to give the number sentence that represented the situation. All students called on came up with the answer: 4 - 4 = 0. She summarized the rule reflected in these examples: ‘‘When numbers before and after the minus sign are the same, the result will be zero.'' She had them practice this rule with five flash cards.

Then, Sun showed the students pictures of an empty plate and a plate with four pears. She asked the class ‘‘How many pears are there altogether on these two plates?'' After two students gave the right answer, Sun called on a girl and asked, ‘‘What method can we use to represent this example?'' The girl replied, ‘‘Subtraction.'' Sun repeated her question by stressing the word ‘‘altogether,'' but the girl still did not get the clue and insisted that subtraction was needed. With obvious frustration, Sun called on a boy who gave the desired answer that addition was needed. She asked the girl to repeat the right answer. Sun did another example of a plate with three apples and an empty plate by using the number sentence, 3 - 0 = 3. Then she summarized the second rule of the lesson, ‘‘When you have a number plus 0 or minus 0, the result will still be the number.''

Next, Sun showed the class two cards with number sentences, 0 + 0 = ___ and 0 - 0 = ___. She called on two students to figure out each answer, and both students answered correctly, saying, ‘‘zero.'' The teacher praised the students and summarized the third rule, ‘‘When you have zero plus zero or zero minus zero, the result is still zero.''

With all the new rules taught, Sun then had students do written exercises from the textbook. She walked around the classroom and helped students whom she thought needed support. She praised students who finished quickly and urged the rest to hurry up. She finished the lesson by asking some students to report their answers to the class.

Sun and Liu's post-lesson conference lasted about 20 minutes and covered seven conversational units in the order shown in Table 1. These units included (1), (3), and (6) related to teaching, (7) related to teaching and students, and (2), (4), and (5), related to teaching, subject matter, and students.

Figure 1 shows that about 64 % of the topics concerned teaching, 22 % of were related to students, and 14 % focused on subject matter. As shown in Figure 2, 59% of the total topics were multifocused topics, 41% connected to teaching, subject matter, and students together, and 18% to both teaching and students. The following is an example of how Liu related teaching, subject matter, and students together as she demonstrates an alternative way to represent the mathematics concept taught in Sun's lesson:

When you teach a number plus zero, you need to show it one by one. A kindergarten teacher had three apples (she put a plate on the board). How many apples on the plate? There are three (she put number 3 under the plate). Then you put another plate and say, ‘‘Now little friends, how many apples on this plate? That is an empty plate. So what number can we use to represent this? We learned just now that we could use zero to represent this (she put number zero under the empty plate). How many apples did the teacher put out all together? Because on the first plate we have three apples and on the second, we have none. If we are going to represent this with a number sentence, what is it?'' Students will say, ‘‘We can use addition to represent this. 3 + 0 = 3 (she puts a number sentence 3 + 0 = 3 under the picture). Let's read the number sentence. What does it mean? It means when three apples plus an empty plate, we still have three apples. Zero represents nothing.'' So, my point is that you need to pay attention to the students' thinking process.

It is striking that Liu dominated the conversation and Sun was almost silent. Figure 3 shows that Liu initiated 94% of the topics while Sun initiated only 6%. The conversation was characterized by a pattern in which the mentor started a unit with several compliments about various aspects of Sun's lesson. Liu then initiated 5 units in which she critiqued various aspects of Sun's lesson and made suggestions followed by more compliments.

Speech-act analysis indicated that compliments, critiques, and suggestions were most prominent. As indicated in Figure 4, 44% of the speech acts were Liu complimenting, 21% critiquing, and 24% making suggestions. Here is an example of how Liu complimented the structure of Sun's lesson at the beginning of the conference:

The structure of your teaching was clear. The lesson was short. However, you were able to teach students complicated concepts step by step, like a number minus itself equals zero. A number plus zero, and a number minus zero equals the number. These concepts are abstract and hard for kids to understand. You were able to teach these in your lesson. The steps of your teaching were clear.

Liu's criticisms were always followed with specific suggestions as shown in Figure 6. About 35% of the time, criticism and suggestion occurred together. Here is an example of how Liu offered criticism along with suggestions for Sun to improve her use of manipulatives in teaching:

I will say what you used in your lesson were teaching manipulatives. I recommend that you use learning manipulatives more. Just now, you showed the students a plate and four apples. I think this process can be reduced. You can ask students to play it out. We need to have them use their own hands and minds in learning. In this way, they will use all of their senses to learn. You need to require your students to do it, and then ask them to report their results to other classmates. Now, you can say, ‘‘Please take out your learning manipulatives. Please put four pieces of wood on your desk. I want you to put all four pieces back in your box. You have four pieces; what do you mean by taking away all? It means to take away all four pieces. What do we have now on the desk? Zero. We have four and take away four. We have zero. Who can use a number sentence to represent this?'' Then you can ask them to use the number cards that you used to represent it. You need to ask them to describe the meaning of this activity. This is the first issue I want to share with you. In future, we really need to pay attention to open up students' minds.''

Finally, Liu was able to articulate reasons, show examples, and do both in presenting most topics. As indicated in Figure 5, 59% of the speech acts were elaborated with both reason and example, 6% were with reason, and another 6% with example, together amounting to 71% of the total speech acts. Only 29% of the speech acts were unelaborated. Here is an example of how Liu presented an issue with both reason and example as she commented on Sun's use of teaching manipulatives in her lesson:

I also think the teaching manipulatives that you made and used are colorful and pretty. However, from scientific and realistic perspectives, you'd better draw a tree instead of a cage. There are three birds in the tree. Our students have pure hearts and are active. They like freedom. So, the cage is not proper here. Actually, the birds you used were simply attached to the cage instead of being placed inside. So in future, when we plan a lesson, we need to pay attention to the scientific and realistic principles of teaching. ‘‘Here is a tree, there are three birds in it and then they all fly away to play. They all fly, fly, and fly away. We had three birds in the tree and all three flew away. What number can we use to represent this?'' I think that is the proper way to use a teaching manipulative. This is only a suggestion for you to consider.

COMPARISON OF TWO CHINESE CASES

Conversational Foci

In both cases, the majority of their exchanges concerned teaching, subject matter, students, and the connections between them. As shown in Figure 1, teaching, subject matter, and students made up 56%, 23%, and 21%, respectively, in the case of Cao and Xue and 64%, 14%, and 22%, respectively, in the case of Liu and Sun. The difference between the two was that Cao and Xue had more subject-matter discussion (23%) than Liu and Sun (14%).

Second, both cases had many multifocused interactions combining teaching, subject matter, and students, although the configurations were somewhat different between the two cases as displayed in Figure 2.

Conversational Forms

The two Chinese cases showed that the mentors initiated more exchanges than their novices (Cao 66% and Liu 94%). The patterns of their conversation were also similar. They started with the mentor complimenting various aspects of novices' lessons followed by a series of critiques and suggestions. A notable difference was that Sun remained almost silent, while Xue responded with agreements, disagreements, and questions. The two cases were similar in that both mentors spent most of their time complimenting, critiquing, and making suggestions for their novices' teaching. In fact, critiques were always followed by suggestions (see Figures 4 and 6). Most speech acts in the two Chinese cases were elaborated with reason or reason and example, although a small number were unelaborated. As shown in Figure 5, 72% of the speech acts in the case of Cao and Xue were elaborated with a reason or with a reason and an example. In the case of Liu and Xue, 65% were elaborated in this manner, with a larger number having both reason and example.

CROSS-NATIONAL COMPARISONS

U.S. AND CHINESE MENTOR-NOVICE CONVERSATIONAL FOCI

The two U.S. cases were clearly different from the two Chinese cases with regard to subject-matter content. Although all four cases paid substantial attention to teaching and student issues, the two Chinese cases had a significant number of topics devoted to issues of subject matter while the two U.S. cases had few. Second, the two U.S. mentor-novice pairs tended to deal with issues of teaching, students, or subject matter in isolation, whereas the two Chinese cases were more likely to discuss these issues in relation to each other as shown in Figure 1. Third, similar findings pertained to the multifocused topics. The U.S. mentors and novices tended to deal with issues of teaching and students with little attention to subject matter. The Chinese mentor-novice pairs were more likely than the U.S. pairs to talk about these issues in relation to one another, often referring to subject-matter issues. This difference is clearly illustrated in Figure 2.

U.S. AND CHINESE MENTOR-NOVICE CONVERSATIONAL FORMS

There are three major differences in the conversational forms of the U.S. and Chinese mentor-novice pairs. First, the U.S. mentors and novices had almost equal opportunities to initiate conversational topics, whereas the Chinese mentors tended to dominate the conversation. Second, the speech acts were differently distributed in the two countries. The U.S. mentors tended to ask questions about their novices' lessons and describe what they saw, while the Chinese mentors were more likely to critique and make suggestions, although they also spent substantial time giving compliments. The U.S. novices tended to explain more in response to mentors' questions than their Chinese counterparts, although they all tended to agree with the mentors. Third, the patterns of exchange between the U.S. Chinese cases were also different. The patterns in the two U.S. cases featured a series of mentors' questions and descriptions about the novices' lessons that were followed by the novices' explanation and agreement. In the two Chinese cases, mentors complimented the novices and then made a series of critiques and suggestions that were sometimes followed by novices' agreement, disagreement, or questions. Fourth, there were some differences in regard to elaboration of speech acts. In most instances the U.S. mentor-novice pairs did not elaborate with reasons or examples, whereas the Chinese pairs tended to elaborate their ideas (Figure 5).

DISCUSSION AND CONCLUSION

What opportunities did the different mentor-novice conversations offer novices in developing professional knowledge necessary for reform-minded teaching? What are the relationships between these mentor-novice interactions and the broad contexts of curriculum and teaching and mentoring in which these mentor-novice conversations are situated? We explore these two questions below.

OPPORTUNITIES FOR LEARNING TO TEACH AND MENTOR-NOVICE INTERACTIONS IN EACH COUNTRY

Teachers' deep understanding of the subject matter that they teach (Ball & Bass, 2001; Ball & McDiarmid, 1989) and their ability to represent such understanding to various groups of students (Grossman, 1990; Shulman, 1987; Wilson & Berne, 1999) are assumed to be crucial to teaching effectively. The analysis of mentor-novice interactions in this study suggests that the opportunities for novices to develop such knowledge were different in each country. In the two U.S. cases, these opportunities were often limited because of an absence of discussion of subject matter and its relation to teaching and students. In contrast, the two Chinese cases paid much more attention to subject matter and also tended to relate it to teaching and students. Research also suggests that effective teaching practice relies heavily on teachers' knowledge about how children of various backgrounds and developmental and intellectual levels learn and about the influences on their learning (Grimmett & MacKinnon, 1992; Ladson-Billings, 1999). Although students were the main focus in both U.S. and Chinese mentor-novice conversations, the two U.S. mentor-novice pairs not only paid more attention to student issues, but they had substantial discussion about individual students' learning and behaviors. In the two Chinese cases, however, students were only discussed in groups or categories. Thus, the U.S. novices had more opportunities to develop knowledge about individual students than their Chinese counterparts did. However, these opportunities were somewhat limited since they often discussed students in isolation and most speech acts were unelaborated. In order to develop teaching effectively, teachers need skills of inquiry to reflect on their teaching practice. They need to be able to pose questions, interpret different situations, develop constructive criticism, and come up with useful ideas to solve various problems in their teaching practice (Cochran-Smith, 1991). The data in this study show that the two U.S. mentor-novice pairs had many opportunities to learn how to question and explain, but fewer opportunities to critique and develop solutions. The Chinese mentor-novice pairs, however, exhibited opposite patterns of interaction.

Finally, another important element of teachers' professional knowledge is the ability to conduct contextualized reasoning about subject matter and pedagogy, arguing the alternatives, and applying their thoughts to uncertain and irregular contexts of teaching (Floden et al., 1990; Kennedy, 1991; Lampert & Clark, 1990; Leinhardt & Greeno, 1986). In the two U.S. mentor-novice conversations, opportunities for novices to develop reasoning skills were limited, since most of their exchanges were unelaborated with no supporting reasons or examples. In contrast, the two Chinese mentor-novice interactions provided several opportunities for novices to learn how to articulate ideas using examples

MENTOR-NOVICE INTERACTIONS AND THE CONTEXTS OF CURRICULUM, TEACHING, AND MENTORING

Recent comparative studies on teachers' instructional practice view teaching and teacher's work as culturally scripted (Hiebert & Stigler, 2000; Stigler, Fermandez, & Yoshida, 1996; Stigler & Hiebert, 1999). Such a cultural script is often reinforced by other contexts of curriculum and of teaching organization. Although only two cases of mentor-novice conversations from each country were analyzed in the present study, these interactions, to some extent, reflect the cultural influences of curriculum and organization of teaching and mentoring in each country.

For example, the substantial focuses on subject matter and the relationship between subject matter, teaching, and children in the two Chinese cases are consistent with a context where elementary teachers are subject-matter specialists. They had to work regularly in a teaching research group based on their subject areas (Wang & Paine, 2003). In addition, their mentors also taught the same subject. These contexts almost certainly influenced the Chinese mentor-novice conversation focus on subject matter. In contrast, the lack of subject-matter focus in the two U.S. cases seems to match the contexts of teaching and mentoring where elementary teachers as well as their mentors are generalists and their subject-matter preparation is often fragmented (Ball, 2000).

We also wonder if such difference, in the attention to subject matter in the two settings exists due to the fact in the U.S. setting, mentors came to the discussions with an agenda that included a focus on one or more specific California teaching standards, thereby possibly reducing the opportunities to focus on subject matter knowledge. However, we were surprised at how few of their interaction topics directly related to the standards in both U.S. cases. A study on mentor-novice discussion in contexts where Chinese teachers work as generalists like the U.S. teachers, such as those in rural and/or small school systems in China, could provide additional information to help verify such interpretation. Moreover, analyzing conversations in the context of other content and linguistic forms between mentors and novices in the U.S. and China, rather than focusing only on lesson-based interactions between mentors and novices could also provide helpful information. Also interesting is that the Chinese pairs focused more on subject matter issues even though they taught at the first and second grade levels. The U.S. pairs taught fifth grade and paid much less attention to subject matter, which seems counterintuitive.

Furthermore, an attention to reason- and example-based critique and suggestions in the two Chinese cases is consistent with the contrived curriculum, teaching organization, and structure of mentoring in place in China. The curriculum provides Chinese teachers with the same goals, content topics, texts, and requirements for their teaching which form a common ground on which mentors can critique the novice teachers' work. The teaching research group in the Chinese school schedules and organizes novices to teach public lessons regularly for colleagues to critique. These contexts allow teachers' instruction to be a public arena where reasonable critiques of and suggestions for one another's teaching are not only possible but also necessary. In addition, mentors were experienced teachers from the same school teaching the same grade level. This gave the mentors some leverage to make contextualized criticism and suggestions for novices. However, inconsistent curriculum material (Cohen & Spillane, 1992) and individualized teaching organization (Feiman-Nemser & Floden, 1986) nurtures a culture of teaching in the U.S. setting in which teachers' selection of goals, coverage of content, and design of teaching strategies are seen as the individual teacher's responsibility. Such a context may make it difficult for the U.S. mentors who are outsiders to the contexts of novices' work to critique what they see and offer suggestions. Thus, it seems reasonable for the U.S. mentors to ask questions about what happened and describe what they saw in the novices' lesson. In this way, mentors can remain respectful of the novices' work while at the same time help them see what they did and what happened in their instruction, allowing them to take necessary actions for themselves. It may also be argued that mathematics teaching may be more open to contextualized reasoning discussions than is the teaching of literacy. Math happened to be the subject area of one Chinese pair, whereas language arts was the subject area for both U.S. pairs. However, our findings were consistent with the findings of a study of Chinese mentor-novice discussions about novices' lessons where teachers of language, physics, and mathematics were involved (Wang & Paine, 2002). Also interesting is that the Chinese mentors were more critical, elaborative in their suggestions and comments about novices' teaching even though the data for these pairs were collected at the beginning of the mentoring program. The U.S. mentors provided less specific and critical feedback to the novices even when conversations occurred at the end of the first year. It seems counterintuitive since the longer mentors work with novices, the more likely one would expect analytical and reflective dialogue.

The stronger attention given to issues of individual students in the U.S. cases as compared to the Chinese cases is difficult to explain given the teaching organizations found in each of the national contexts. This finding, though, is consistent with those based on interviews with more mentors and novices from China, the U.S., and England (Wang, 2001). What is puzzling here is that mentors in the U.S. who were outsiders to novices' actual teaching contexts and who might have relatively little deep understanding about novices' students, devoted more attention to individual students. Chinese mentors, on the other hand, who were insiders in novices' teaching contexts since they worked at the same grade level and in the same school as the novices, a context that may have allowed them more opportunities to discuss issues of individual students, did the opposite in their interactions with students. This may, in part, reflect a common preference in U.S. educational circles away from stereotyping cultural groups while the larger classes that Chinese teachers have to teach may prevent them from focusing on individual students. It is worth noting that these interpretations were made based on only one post-lesson interaction from each case. Such interpretation surely deserves further verification and alternative interpretations when various mentor-novice interactions and more mentor-novice cases in different teaching and mentoring contexts are carefully studied. Thus, readers should be cautious in generalizing these interpretations to the larger contexts of teacher mentoring in both countries.

QUESTIONS AND IMPLICATIONS

The findings in this study also raise questions about and offer implications for two popular assumptions about novices' learning to teach with mentors. First, it is widely assumed in the western literature that novices should find their own voice as young professionals as they learn to teach with mentors. This is regarded as an important step in becoming a professional teacher (Bullough, 1990; Corley, 1998; Kilbourn & Roberts, 1991; Schmidt & Knowles, 1994). Consistent with this line of thinking is the assumption that any kind of contrived collegiality, where colleagueship and partnership are administratively structured and imposed, presents difficulties for the creation and persistence of a collaborative relationship among teachers (Cochran-Smith & Paris, 1995; Hargreaves, 1984). Such an assumption, on the one hand, reinforces the popular focus of research in teacher learning on the forms of teacher interactions where novices have autonomy in developing their own voices and fortifies the practice of professional development that centers on novices' feelings, confidence, and ideas. On the other hand, it leaves what novices need to learn in a state of ambiguity, and limits alternative environments in which they can learn what they need to learn.

Our findings challenge such an assumption by showing that the U.S. novices' equal participation in the dialogue with their mentors did not necessarily provide important opportunities for novices' learning to teach. The Chinese teachers' unequal participation in the dialogue with mentors did not exclude them from accessing the constructive criticisms, reasonable compliments, and useful suggestions for their teaching. As a side note, even the more silent novice, Sun, over the course of the year, made the most progress in learning to teach mathematics for understanding and problem solving with the support of her mentor in a didactic mentoring situation as documented elsewhere (Wang & Paine, 2001). In addition, the contrived curriculum and teaching organization did not necessarily prevent Xue from showing her disagreement with and raising questions about what the mentor said. An implication from these findings is that we should not only focus on novices' developing their own voices, but we should also focus on what they talk about and how they approach the issues of teaching when we develop teacher mentoring relationships and support mentor-novice collaboration.

Second, much of the consideration in U.S. teacher mentoring programs has been given to providing opportunities, time, and resources for mentors and novices to talk about their instruction. The underlying assumption of this focus is that once mentor and novice have opportunities to observe and discuss teaching, novice teachers will be able to learn to teach effectively. This study suggests that effective mentor-novice interactions are not only a function of opportunity, time, and resources for mentor and novice to observe and talk, but also a function of the broader curriculum and teaching contexts in which the mentor-novice relationship is situated. This finding offers two implications for teacher mentoring programs and policy. First, it is important for us to help both mentors and novices identify the influences of the broader contexts of curriculum and teaching on their relationships and interactions as we develop mentoring programs and facilitate support for mentor-novice interactions. Second, teacher mentoring alone may not be fully effective in supporting novice teachers movement toward reform-minded teaching practice, when it is not an integral part of the larger effects that transform the broader contexts of teaching and schooling.

Notes

1 All the names used here or in the other places related to this paper are pseudonyms.

2 Normal school is a special high school that draws students from junior secondary schools based upon their test scores on the junior secondary school graduation examination at the provincial level. Students in normal schools are educated to be elementary teachers where they study both general secondary courses like those offered in high school and teacher education courses.

3 Chinese elementary teachers are often assigned to teach the same subject matter in two or more of the same grade level classes like the middle and high school teachers in the United States but they go to a different classroom to teach.

4 The Effects of Mentoring on New Teacher Development and Student Outcomes examined the conditions of mentoring that most affect teacher and student development over the first two years of a teacher's career. It observed beginning teachers' instruction and mentor-novice teacher' interaction, interviewed mentor and novice teachers, and collected standardized test scores of students.

5 The Learning from Mentors project collected data from 23 mentor-novice pairs in both induction and preservice programs in three countries, the United States, England, and China. The study was designed to understand what novices learn, how mentors assist novices' learning, and how the contexts of mentoring influence novices' learning and mentoring.

References

Achinstein, B., & Villar, A. (2002, April). The politics of the mentoring process for beginning teachers: Tensions and negotiations. Paper presented at the Annual Meeting of the American Educational Association, New Orleans.

Apple, M. W. (2001). Markets, standards, teaching, and teacher education. Journal of Teacher Education, 52(3), 182–196.

Austin, J. L. (1962). How to do things with words. Oxford, UK: Oxford University Press.

Austin, T., & Fraser-Abder, P. (1995). Mentoring mathematics and science preservice teachers for urban bilingual classrooms. Education and Urban Society, 28(1), 67–89.

Bakhtin, M. M. (1986). Speech genres and other late essays. Austin: University of Texas Press.

Ball, D. L. (2000). Bridging practices: Intertwining content and pedagogy in teaching and learning to teach. Journal of Teacher Education, 51(3), 241–247.

Ball, D. L., & Bass, H. (2001). Interweaving content and pedagogy in teaching and learning to teach: Knowing and using mathematics. In J. Boaler (Ed.), Multiple perspectives on mathematics teaching and learning (pp. 83–104). Westport, CT: Ablex Publishing.

Ball,D. L., &McDiarmid, G.W.(1989). The subject matter preparation of teachers (Issue Paper 89-4). East Lansing: National Center of Research on Teacher Learning, Michigan State University.

Beasley, K., Corbin, D., Feiman-Nemser, S., & Shank, C. (1996). ‘‘Making it happen'': Teachers mentoring one another. Theory into Practice, 35(3), 158–164.

Berliner, D. C., & Biddle, B. J. (1996). Standards amidst uncertainty and inequality. School Administrator, 53(5), 42–44, 46.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Education Researcher, 18(1), 32–34.

Bullough, R. V., Jr. (1990). Supervision, mentoring, and self-discovery: A case study of a first- year teacher. Journal of Curriculum and Supervision, 5(4), 338–360.

Carter, K. (1990). Teachers' knowledge and learning to teach. In W. R. Houston (Ed.), Handbook of research on teacher education (pp. 314–338). New York, NY: MacMillan.

Cochran-Smith, M. (1991). Learning to teach against the grain. Harvard Educational Review, 61(3), 279–310.

Cochran-Smith, M., & Fries, M. K. (2001). Sticks, stones, and ideology: The discourse of reform in teacher education. Educational Researcher, 30(8), 3–15.

Cochran-Smith, M., & Lytle, S. (1999). Relationship of knowledge and practice: Teacher learning in communities. Review of Research in Education, 24, 249–298.

Cochran-Smith, M., & Paris, P. (1995). Mentor and mentoring: Did Homer have it right? In J. Smith (Ed.), Critical discourses on teacher development (pp. 181–202). London, UK: Cassell.

Cohen, D. K., & Spillane, J. P. (1992). Policy and practice: The relations between governance and instruction. Review of Research in Education, 18, 3–49.

Corley, E. L. (1998, October). First-year teachers: Strangers in strange lands. Paper presented at the Annual Meeting of the Midwestern Educational Research Association, Chicago.

Darling-Hammond, L., & Ball, D. L. (1998). Teaching for high standards: What policymakers need to know and be able to do (CPRE Joint Report Series No. JRE-04). Philadelphia: National Commission on Teaching and America's Future Consortium for Policy Research in Education.

Elbaz, F. (1983). Teacher thinking: A study of practical knowledge. London, UK: Croom Helm.

Feiman-Nemser, S. (2001). Helping novices learn to teach: Lessons from an experienced support teacher. Journal of Teacher Education, 52(1), 17–30.

Feiman-Nemser, S., & Floden, R. (1986). The culture of teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 505–526). New York: Macmillan.

Feiman-Nemser, S., & Parker, M. B. (1990). Making subject matter part of the conversation or helping beginning teachers learn to teach (Research Report 90-3). East Lansing, MI: National Center for Research on Teacher Education, Michigan State University.

Feiman-Nemser, S., & Parker, M. B. (1993). Mentoring in context: A comparison of two U. S. programs for beginning teachers. International Journal of Educational Research, 19(8), 699–718.

Floden, R. E., Klinzing, H. G., Lampert, M., & Clark, C. (1990). Two views of the role of research on teacher thinking (Paper 90-4). East Lansing, MI: National Center for Research on Teacher Education, Michigan State University.

Grimmett, P., & MacKinnon, A. (1992). Craft knowledge and the education of teachers. Review of Research in Education, 18, 385–456.

Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press.

Hargreaves, A. (1984). Experience counts, Theory doesn't: How teachers talk about their work. Sociology of Education, 57(4), 244–254.

Hiebert, J., Gallimore, R., & Stigler, J. W. (2002). A knowledge base for the teaching profession: What would it look like and how can we get one? Educational Researcher, 31(5), 3–15.

Hiebert, J., & Stigler, J. W. (2000). A proposal for improving classroom teaching: Lessons from the TIMSS video study. Elementary School Journal, 101(1), 3–20.

Holmes Group. (1986). Tomorrow's teachers. East Lansing, MI: Author.

Holmes Group. (1990). Tomorrow's schools. East Lansing, MI: Author.

Huling-Austin, L. (1992). Research on learning to teach. Journal of Teacher Education, 43(3), 173–180.

Interstate New Teachers Assessment and Support Consortium. (1992). Model standards for beginning teacher licensing and development: A resource for state dialogue: Washington, DC: Council of Chief State School Officers.

Kennedy, M. (1991). An agenda for research on teacher learning (NCRTL Special Report). East Lansing: National Center for Research on Teacher Learning, Michigan State University.

Kilbourn, B., & Roberts, G. (1991). May's first year: Conversations with a mentor. Teachers College Record, 93(2), 252–264.

Klug, B. J., & Salzman, S. A. (1991). Formal induction vs. informal mentoring: Comparative effects and outcomes.. Teaching and Teacher Education, 7(3), 241–251.

Ladson-Billings, G. (1999). Preparing teachers for diversity: Historical perspectives, current trends, and future directions. In L. Darling-Hammond & G. Sykes (Eds.), Handbook of policy and practice (pp. 86–124). San Francisco: Jossey Bass.

Lampert, M., & Clark, C. M. (1990). Expert knowledge and expert thinking in teaching: A response to Floden and Klinzing. Educational Researcher, 19(5), 21–23.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

Leinhardt, G., & Greeno, J. G. (1986). The cognitive skill of teaching. Journal of Educational Psychology, 78(2), 75–95.

Lewis, C. C. (2000). Lesson study: The core of Japanese professional development. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.

Lewis, C. C., & Tsuchida, I. (1998). A lesson is like a swiftly flowing river: How research lessons improve Japanese education. American Educator, 22(4), 12–17, 50–52.

Little, J. W. (1990a). The mentoring phenomenon and the social organization of teaching. Review of Research in Education, 16, 252–279.

Little, J. W. (1990b). The persistence of privacy: autonomy and initiative in teachers' professional relationship. Teachers College Record, 91(4), 509–536.

Ma, L. (1999). Knowing and teaching elementary mathematics. Mahwah, NJ: Erlbaum.

National Council for Accreditation of Teacher Education. (1998). Program standards for elementary teacher preparation, Review and comment edition. Washington, DC: Author.

National Council for Accreditation of Teacher Education. (1999). Proposed NCATE 2000 unit standards. Washington, DC: Author.

National Council for the Social Studies. (1994). Expectations of excellence: Curriculum standards for social studies. Washington, DC: Author.

National Council of Teachers of English and the International Reading Association. (1996). Standards for the English language arts. Urbana, IL: Author.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.

National Research Council. (1996). National science education standards. Washington, D.C: National Academy Press.

Odell, S. J., & Ferraro, D. P. (1992). Teacher mentoring and teacher retention. Journal of Teacher Education, 43(3), 200–204.

Odell, S. J., & Huling, L. (Eds.). (2000). Quality mentoring for novice teachers. Indianapolis, IN: Kappa Delta Pi.

Paine, L. (2001). Outside/insider talk about teaching: Supporting teacher learning in communities of practice. Paper presented at the 45th Annual Meeting of Comparative and International Education Society, Washington, DC.

Richardson, V. (1996). The role of attitude and beliefs in learning to teach. In J. Sikula, T. Buttery & E. Guyton (Eds.), Handbook of research on teacher education (2nd ed., pp. 102–119). New York: Macmillan.

Rogoff, B. (1984). Introduction: Thinking and learning in social contexts. In B. Rogoff & J. Lave (Eds.), Everyday cognition: Its development in social context (pp. 1–8). Cambridge, MA: Harvard University Press.

Schmidt, M., & Knowles, J. G. (1994, April). Four women's stories of ‘‘failure'' as beginning teachers. Paper presented at the Annual Conference of American Educational Research Association, New Orleans, LA.

Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–22.

Stigler, J. W., Fermandez, C., & Yoshida, M. (1996). Traditions of school mathematics in Japanese and American elementary classrooms. In L. P. Steffe & P. Nesher (Eds.), Theories of mathematical learning (pp. 149–175). Mahwah, NJ: Erlbaum.

Stigler, J. W., & Hiebert, J. (1999). Teaching gap. New York: Free Press.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. New York: Sage.

Strong, M., & Baron, W. (in press). An analysis of mentoring conversations with beginning teachers: Suggestions and responses. Teaching and Teacher Education.

Strong, M., & St. John, L. (2001). A study of teacher retention: The effects of mentoring for beginning teachers (Research Working Paper #3). Santa Cruz: New Teacher Center, University of California, Santa Cruz.

Sweeny, B., & DeBolt, G. (2000). A survey of the 50 states: Mandated teacher induction programs. In S. Odell & L. Huling (Eds.), Quality mentoring for novice teachers (pp. 97–106). Indianapolis, IN: Kappa Delta Pi.

Tharp, G., & Gallimore, R. (1988). The intra psychological plane of teacher training: Internalization of high order teaching skills. In Tharp & Gallimore (Eds.), Rousing mind to life: Teaching, learning, and schooling in social context (pp. 217–248). Cambridge, UK: Cambridge University Press.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Vygotsky, L. S. (1994). The problem of the environment. In Rene van der Veer & J. Valsiner (Eds.), The Vygotsky reader (pp. 338–354). Cambridge, MA: Blackwell.

Wang, J. (2001). Contexts of mentoring and opportunities for learning to teach: A comparative study of mentoring practice. Teaching and Teacher Education, 17(1), 51–73.

Wang, J., & Odell, S. J. (2002). Mentored learning to teach and standards-based teaching reform: A critical review. Review of Educational Research, 7(3), 481–546.

Wang, J., & Paine, W. L. (2001). Mentoring as assisted performance: A pair of Chinese teachers working together. The Elementary School Journal, 102(2), 157–181.

Wang, J., & Paine, L. (2002, April). Forms and substances of mentor-novice interactions on novices' teaching: Lessons from Chinese readers. Paper presented at the Annual Meeting of the American Educational Association, New Orleans.

Wang, J., & Paine, L. W. (2003). Learning to teach with mandated curriculum and public examination of teaching as contexts. Teaching and Teacher Education, 19(1), 75–94.

Wilson, S. M., & Berne, J. (1999). Teacher learning and the acquisition of professional knowledge: An examination of research on contemporary professional development. Review of Research in Education, 24, 173–209.

 Cite This Article as: Teachers College Record Volume 106 Number 4, 2004, p. 775-813

http://www.tcrecord.org ID Number: 11535, Date Accessed: 2/14/2006 11:48:33 AM

by Frederick M. Hess & Andrew P. Kelly — 2007

INTRODUCTION

School leadership is the key to school improvement. School principals are the front-line managers, the small business executives, the team leaders charged with leading their faculty to new levels of effectiveness. In this new era of educational accountability, where school leaders are expected to demonstrate bottom-line results and use data to drive decisions, the skill and knowledge of principals matter more than ever. The rise of charter schooling, increasing school choice, and more flexible teacher compensation and hiring have granted thousands of principals new opportunities to exercise discretion and operate with previously unimagined leeway. In this environment, school improvement rests to an unprecedented degree on the quality of school leadership.

Superintendents make clear that they hold new and more demanding expectations for principals. Public Agenda notes that when today's superintendents "describe what they are trying to accomplish" they use the words "accountability, instructional leadership, closing the achievement gap, [and] teacher quality" (Farkas, Johnson, & Duffett, 2003, p. 22). However, principals themselves suggest that they are not fully equipped for all of the challenges they face. Just 36 percent of principals report that tougher scrutiny of teachers is resulting in denied tenure for weak teachers and just 30 percent reported that students' assessed performance is being factored into the evaluation of teachers (Farkas et al., 2003, p. 21).

In this changing context, an array of scholars have asked whether traditional approaches to preparing and licensing principals are sufficient (Elmore, 2000; Thomas B. Fordham Foundation, 2003; Hess, 2003; Murphy, 2001a; Tucker, 2003). Leaders of the University Council for Education Administration have asserted that "in order to build programs that support leadership for learning—we must rethink and revise our practice in several areas" (Young & Kochan, 2004, p. 121). Theodore Kowalski (2004), an influential scholar of educational administration, has advocated "substantial reforms in administrator preparation, program accreditation, and state licensing standards" (p. 93).

Principals themselves are among the first to agree that they need to be more effectively prepared for their jobs. All but 4 percent of practicing principals report that on-the-job experiences or guidance from colleagues has been more helpful in preparing them for their current position than their graduate school studies. In fact, 67 percent of principals reported that "typical leadership programs in graduate schools of education are out of touch with the realities of what it takes to run today's school districts" (Farkas et al., 2003, p. 39).

A recent four-year study by the president of Teachers College at Columbia University, Arthur Levine (2005), raised the stakes in this debate by harshly assessing the quality of educational administration programs. Based on a survey of practicing principals and education school deans, chairs, faculty, and alumni, as well as case studies of 25 school leadership programs, Levine (2005) concluded that "the majority of [educational administration] programs range from inadequate to appalling, even at some of the country's leading universities" (p. 23). In particular, he found that the typical course of studies required of principal candidates was largely disconnected from the realities of school management, though Levine did not seek to analyze the content of these courses. In light of the Levine analysis, and given the increasing demands on school leaders, the question of what candidates are actually being taught in principal preparation has taken on heightened significance.

Unfortunately, there exists no systematic information addressing this question. In this study, we set out to examine what candidates are taught in the core courses that constitute principal preparation at established principal-preparation programs. In doing so, we hope to provide a first, imperfect, examination of this important topic.

This question includes a range of more specific queries and concerns, including:

• Are principals taught the fundamentals of management?

• Are principals trained in instruction and pedagogy?

• Is there evidence of a clear ideological direction in instruction?

• Is there notable variation among the preparation offered in the most prestigious, the largest, and more typical institutions?

Absent a clear understanding of just what principals are learning in the course of their preparation, debates about how we should change principal preparation or about the importance of preparation and licensure rest more on faith than fact. Inevitably, we approach the analysis with certain conceptions of the skills and knowledge that contemporary principals require. We believe that effective principal preparation ought to include considerable attention to accountability, managing with data, and utilizing research; to hiring, recruiting, evaluating, and terminating personnel; to overseeing an effective instructional program; and to exposing candidates to diverse views regarding educational and organizational management. The findings here do not provide clear-cut prescriptions as to what programs should teach, but can raise thought-provoking questions for educational administration researchers, practitioners, and policy-makers seeking to improve existing arrangements.

We examined the course units and required readings contained in 210 syllabi collected from a national cross-section of 31 principal-preparation programs. While some in the professional education community have argued that syllabi can tell us nothing about a course or, as David Labaree has suggested, are nothing more than "ideological portraits" (quoted in Keller, 2003, p. 8), this study presumes that university syllabi generally reflect the content and perspective of the courses being taught. While syllabi cannot convey the tone of classroom instruction, they enumerate what topics professors will cover and what students will read. Ultimately, this study rests on the notion that syllabi are like blueprints: they reveal structure and design, even if they do not fully reflect what real-life instruction looks like. Recognizing that blueprints necessarily lack context, we sought to avoid relying upon simple word counts. Rather, we gauged the emphasis of each lesson and coded each into one of seven areas of principal competency. Within each area, we then coded the various lessons based on their primary focus. This two-step approach allowed us to provide a broad take on the curricular landscape and to explore particular topics in some detail.

EXISTING RESEARCH

Almost no current research systematically documents the content studied in the nation's principal-preparation programs, the instructional focus, or the readings assigned to students. Beyond the 2005 Levine study, recent research and commentary has focused on the need to reshape the principal's role so that school leaders are more focused on increasing student achievement, driving school improvement, and meeting the challenges of standards-based accountability and charter schools (Grogan & Andrews, 2002; Portin, Schneider, DeArmond, & Grundlach, 2003). In this study, we document the attention devoted to seven areas of principal responsibility, each of which have been deemed vital to effective school leadership by at least some leading thinkers in the field. The seven are: managing for results, managing personnel, technical knowledge, external leadership, norms and values, managing classroom instruction, and leadership and school culture. Clearly, there are other ways to frame this list of skills and other skills that might be included, but we believe this a useful and constructive rubric for examining the attention devoted to important areas of knowledge.

Scholars of educational leadership have highlighted the importance of each of these seven categories. First, the "managing for results" category, as envisioned for this study, mirrors the skills set forth by Tucker and Codding (2002). They suggested that preparation should stress the "principal's role as the driver for results" and highlight "the crucial role of data in the drive for results, from the careful setting of targets to the collection, display, and analysis of implementation and outcome data to the use of data for setting goals, monitoring progress, allocating and reallocating resources, and managing the school program" (p. 37).

Second, the principal's role in "managing personnel" has taken on new significance as the pressures of accountability increase the expectations on school leaders to hire, induct, and evaluate personnel in a sensible manner. As the most commonly assigned human resources text states, "School districts are ethically bound to find the most talented and skilled people available to achieve their mandate of educating children" (Rebore, 2004, p. 93). Recent survey data from Public Agenda reveals that 78 percent of superintendents and 57 percent of principals believe that principals are evaluated based on their ability to "judge and improve teacher quality" (Farkas et al., 2003, p. 21).

Third, the longstanding emphasis on "technical knowledge" of school law, school finance, and facilities management in principal preparation has fallen out of favor of late. As Ferrandino and Tirozzi (2004) recently put it, "Yesterday's principal was often a desk-bound, disciplinarian building manager who was more concerned with the buses running on time than academic outcomes." Nonetheless, teaching aspiring principals this body of vocational knowledge remains an identifiable, significant element of instruction. Recent survey data reveal that 82 percent of principals report dealing with school facilities, resources, or procedures every day while only 40 percent say the same about driving student achievement ("Leading for Learning", 2004).

Fourth, issues of external leadership loom large in the educational administration discourse. Scholars of educational leadership have pointed out the importance of dealing with external constituencies and attending to school board relations, school-community partnerships, and school politics (Bagin & Gallagher, 2001; Kowalski, 1995). As Hoy and Miskel (2005) point out in one commonly assigned leadership text, "understanding the existing and budding environmental influences is of extreme importance to school administrators" (p. 241).

Fifth, educational administration thinkers and scholars have long argued for the centrality of "norms and values" in promoting equitable and effective schooling (Sergiovanni, 1992; Ryan & Bohlin, 1999; Meier, 1995; Cochran-Smith, 2004). Marilyn Cochran-Smith (2004) asserts that because schooling is inherently political, "participants" in the educational process "deliberately claim the role of educator as well as activist based on political consciousness and on ideological commitment to diminishing the inequities of American life" (p. 19).

Finally, receiving less attention in this analysis—in part because they are the most commonly discussed elements of leadership preparation—are the topics "instructional leadership" and "school culture." Instructional leadership, in particular, tends to encompass the many different facets of school management (DuFour, 2002; King, 2002; Murphy, 2001b Supovitz & Poglinco, 2001; Elmore, 2000). For the purposes of this study, we have adopted a focused conception of instructional leadership that emphasizes matters of pedagogy, curriculum, and classroom management.1 With regard to culture, leading scholars have observed, "School culture affects every part of the enterprise from what faculty talk about in the lunch room, to the type of instruction that is valued . . . to the importance of learning for all students" (Peterson & Deal, 1999, p. 7). While both sets of issues are important, this study focuses more upon the prevalence of skills that are newly significant in educational leadership.

To date, existing research has not scrutinized the attention devoted to each of these vital management questions in the course of principal preparation. The only previously published study to explore the content of administrator preparation using course syllabi was conducted by Nicolaides and Gaynor (1992). The authors analyzed 36 syllabi from doctoral programs at the 37 University Council for Educational Administration (UCEA) programs to examine the focus of administrative theory courses and isolated five basic themes: theoretical and historical foundations, process and change, sociopolitical structures, leadership, and culture and symbols. A related effort by Norton and Levan (1987) surveyed UCEA doctoral programs and found that more than 60 percent of content addressed managing personnel, school administration, and technical knowledge of law and finance.

Two recent studies, conducted by Steiner and Rozen (2004) and Butin (2004), have examined the content of syllabi in teacher-preparation programs. Steiner's analysis examined 165 required-course syllabi from 14 elite teacher-preparation programs and two other programs, focusing specifically on the areas of foundations, reading instruction, and math instruction. Butin analyzed 89 education foundations syllabi from 85 teacher-preparation programs that had syllabi posted on the Internet. Steiner argued that the data showed that teacher-preparation courses were ideologically biased, while Butin disputed Steiner's finding.

The field of educational leadership has suffered from a general dearth of systematic scholarly inquiry. Leading authorities have pointedly observed that the overall landscape of educational administration research is "considerably bleaker than most would prefer" (Murphy & Vriesenga, 2004, p. 11).2 In particular, educational administration scholars have termed the body of research on administrator preparation "scant" (Lashway, 2003). For instance, a recent effort to analyze the state of administrator preparation conducted by the National Commission for the Advancement of Educational Leadership Preparation (NCAELP) commissioned six papers which yielded essays on topics like the challenges of reforming administrator preparation (Young, Peterson, Short, 2002), the need to rethink the foundations of leadership preparation (Murphy, 2001b, promising training programs across the country (Jackson & Kelley, 2002), and a "self-evaluation" for preparation programs (Glasman, Cibulka, & Ashby, 2002). While useful, the NCAELP effort did not seek to present systematic data regarding what preparation programs do or what they teach.

The larger body of literature on educational administration preparation reflects the NCAELP effort; it consists primarily of essays or anecdotal descriptions of particular programs. According to Murphy and Vriesenga (2004), between 1975 and 2002, 296 articles on any facet of administrator preparation were published in academic journals. Of these, just 81 (or 27.4%) were empirical in some sense, and just 19 of those addressed any element of administrator preparation curricula.

Most of these 19 articles covered a particular curricular domain like technology (Garland, 1990, Bozeman & Spuck, 1991 , diversity (Parker & Shapiro, 1992; Herrity & Glasman, 1999), social justice (Lyman & Villani, 2002), counseling aptitude (Lampe, 1985), supervision and decision-making skills (Sweeney & Moeller, 2001; Roberts, 1991), or special education (Hirth & Valesky, 1990; Sirotnik & Kimball, 1994).3 The only two that were similar to the present inquiry have already been discussed above. Ultimately, Murphy and Vriesenga (2004) have concluded, "From the extant research, we know almost nothing about the traditional curricular domains of preparation programs . . . nor . . . the shape of curriculum in a post-theory era where issues around teaching and learning and community are reshaping the profession" (p. 24).

METHODS

This study sought to examine what skills and knowledge are being taught in principal-preparation programs and how that material was being approached.4 Ultimately, we were able to collect 210 syllabi from 31 programs. Data collection, coding, and analysis took place between February and December 2004, and participating scholars and institutions were promised anonymity. In all cases, we collected the most recently available syllabi from each institution. In the handful of cases where programs indicated that old syllabi were obsolete but new syllabi had not yet been constructed, no syllabi were collected.

Sample

In early 2004, The U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS) listed 496 administrator-preparation programs. This study drew its sample from three categories of those programs: influential elite programs, large programs that train the most candidates, and more typical programs. This sampling strategy was designed so as to avoid the criticisms that have been made of the Steiner and Rozen (2004) study of teacher preparation syllabi which looked primarily at elite preparation programs, said by critics to be atypical of standard practice. Our sampling strategy allowed us to see whether practices varied between elite and non-elite, or between large and small programs.

The pool of elite programs included the 20 ranked by U.S. News & World Report in 2004 as the nation's top administration programs (two schools were tied for 20th position). Large programs included the 20 educational leadership preparation programs that awarded the largest number of M.Ed. degrees as reported by IPEDS in 2003.5 IPEDS is the only national source available on completion of administrator-preparation programs, and the 2003 data was the most recent available when the study commenced. A third group of 20 programs was randomly drawn from the remaining universe of IPEDS institutions.

From this initial pool of 61 programs, 56 qualified for analysis. The 21 "elite" programs yielded a total of 19. One program was excluded because it does not have a principal-preparation program and a second because it operates an unorthodox accelerated licensure track that does not have courses with syllabi. The 20 largest programs yielded 17 eligible for analysis. One program was incorrectly included in the IPEDS rankings. A second was dropped because its M.Ed. in educational leadership does not lead to licensure or certification in any state and requires candidates to take additional courses at other institutions to fulfill state guidelines. A third program offered only a nontraditional certification program without conventional courses.

From this sample of 56 eligible programs, we were able to collect at least four "core" course syllabi amenable to analysis from 31 (55% of the entire sample of 56).

Of the 19 eligible elite programs, we were able to retrieve at least four "core" syllabi from 13. Of the 17 eligible "large" programs, we obtained at least four syllabi from 11 that met our standards for coding. Of the approximately 450 other preparation programs, we randomly selected 20 for inclusion in the third group. Of these, we collected at least four syllabi from 12 of the programs, though only seven sets of syllabi met the standards for systematic coding.

Syllabi selection

For each program, we sought to collect syllabi for the courses that constituted the core of the principal-preparation program. For those schools that did not have a clearly delineated principal-preparation track, we contacted the institutions to determine which courses were required of principal candidates.

Generally, programs had five to ten core courses that were required of all aspiring principals. Beyond that, some programs required candidates to complete unspecified electives, while others offered candidates a menu of classes from which to choose. In the case of unspecified electives, we did not seek additional materials. In cases where schools required students to take a set number of electives from a specific list of eligible classes, we randomly sampled a number of syllabi from the list equal to the number of required electives.

The final analysis only includes programs from which we collected at least four syllabi from core courses and the specified menu of electives. Consequently, the analysis includes at least four syllabi from each of the 31 programs.

Syllabi collection

The retrieval process involved three steps. First the schools' websites were searched for target syllabi that were available online. At the handful of institutions where the majority of core course syllabi were available online, individual professors who taught the remaining courses were contacted to request additional syllabi. Each faculty member was contacted three times to request their participation.

At the vast majority of institutions, where few or none of the school's syllabi were available online, we contacted the department via email to request the materials. For schools that did not present a clearly delineated list of principal-certification classes, the email requested a list of required courses and the corresponding syllabi. Each program was contacted at least eight times. Occasionally, department chairs deferred to the individual professors to provide their syllabi. In these cases, individuals were contacted at least three times to request their cooperation.

Standards for including syllabi

We were able to collect 243 syllabi in accord with these collection rules.6 To be included in the sample, syllabi had to provide a clear course outline that enumerated specific topics for each week, class, or unit. Syllabi that listed only "course objectives" or that lacked any obvious weekly structure were not included. Ultimately, these restrictions dictated that we discard 33 deficient syllabi, producing a final total of 210 syllabi. Table 1 outlines how the collected syllabi and course weeks were distributed in terms of the initial sample frame.

Table 1: Distribution of Collected Syllabi and Course Weeks

Program Type Eligible Programs Programs Providing 4+ Suitable Syllabi Total Syllabi Total Course Weeks

Elite 19 13 84 972

Large 17 11 78 827

Typical 20 7 48 625

Total 56 31 210 2,424

Coding

The various course weeks of instruction were coded according to the seven categories: managing for results, managing personnel, technical knowledge, external leadership, norms and values, managing classroom instruction, and leadership and school culture. The handful of weeks that surveyed multiple topics or that encompassed other specialized topics that did not fit this rubric were coded "other".

The "managing for results" category encompassed course weeks dedicated to school-level program implementation, evaluation, and organizational change efforts that require an active principal role. The specific focus of the weeks that comprise this category typically included issues like "accountability," "evaluation," "assessments," "data management," "decision-making," "strategy," "organizational structure," and "change."

The weeks coded "managing personnel" entailed any course weeks that dealt with a principal's relations with school employees, primarily teachers but also assistant principals, specialists, and staff members. These weeks discussed issues like "recruitment," "selection," "induction," "teacher evaluation," "clinical supervision," "motivation," "conflict management," "professional development," and "termination" or "dismissal."

The "technical knowledge" category entailed all weeks that dealt with instruction on matters of law, finance, facilities, data and research training, or technology. These course weeks discussed "school funding," "budgeting," "due process," "church and state," "student and teacher freedoms," "tort law," "literature reviews," "sampling," "statistical analyses," and "database management."

Those weeks coded as addressing "external leadership" dealt with the instruction that focused on the principal's relations with external constituents. Lessons that covered school board relations, collective bargaining, public relations and marketing, parent and community relations, and politics and policy were coded as external leadership.

The "norms and values" category entailed weeks that exposed principal candidates to different educational and pedagogical philosophies, discussed debates about the nature and purpose of public schooling, and examined the racial, ethnic, and socioeconomic context of education (the sociology of education). These weeks typically included lessons on "stratification," "multiculturalism," "diversity," "constructivism," "inequality," "equity," "social justice," and "gender."

The "leadership and school culture" weeks encompassed those lessons that discussed leadership theory and the principal's role in cultivating school culture. Typical topics in this category included "the frames of leadership," "symbolic leadership," "leadership versus management," "creating a school culture," "leading with vision," and "school climate."

Weeks coded as "managing classroom instruction" dealt with the principal's role and influence on what occurred in the classroom. Course weeks focused on "curriculum," "learning theories," "instructional leadership," "pedagogy," "classroom management," and "collaborative learning" were coded as managing classroom instruction.

FINDINGS

The primary unit of analysis for this study is the "course week"—or what principal candidates primarily studied during a week in a given course. Because the 210 syllabi averaged more than ten course weeks apiece, analysis of the full sample yielded a total of 2,424 course weeks.

The initial review sought to examine the amount of attention paid to the various areas of leadership across all coded syllabi. Table 2 shows the relative attention (measured in course weeks) that principal-preparation courses paid to the seven major strands of school leadership outlined above. Programs devoted more than a quarter of their time to technical knowledge, about 15 percent to managing for results and managing personnel, and less to other areas.

Table 2: What Leadership Programs Cover

School Type Managing for Results Managing Personnel Technical Knowledge External Leadership Norms and Values Leadership and School Culture Managing

Classroom

Instruction Survey/

Other

Elite 17.1%

(166/972) 13.3%

(129/972) 24.9%

(242/972) 9.5% (92/972) 14.9%

(145/972) 7.3%

(71/972) 11.8%

(115/972) 1.2%

(12/972)

Large 12.9%

(107/827) 15.2%

(126/827) 36.0%

(298/827) 7.1%

(59/827) 8.1%

(67/827) 4.1%

(34/827) 12.0 %

(99/827) 4.5%

(37/827)

Typical 17.3%

(108/625) 16.8%

(105/625) 28.5%

(178/625) 6.9%

(43/625) 13.0%

(81/625) 6.6%

(41/625) 8.0%

(50/625) 3.0%

(19/625)

Total 15.7%

(381/2424) 14.9%

(360/2424) 29.6%

(718/2424) 8.0%

(194/2424) 12.1%

(293/2424) 6.0%

(146/2424) 10.9%

(264/2424) 2.8%

(68/2424)

Striking in Table 2 are the broad similarities across the three kinds of programs. While the critiques of the Steiner study said that elite education school programs were atypical of common practice, we find little evidence of systematic variation among programs in the kinds of topics they address. There are noticeable differences in emphasis, but they are not nearly as distinct as some observers might have anticipated. While the figures in Table 2 should be interpreted with caution, they provide a rough sense of how time was allocated.

Managing for results

"Managing for results" included those weeks linking school management with issues of quality control, improved performance, and rethinking or restructuring practices and routines. Weeks addressing managing for results focused on school-level implementation, evaluation, and organizational change efforts that require an active principal role. In general, about 16 percent of course weeks were devoted to managing for results.

In an era of No Child Left Behind and aggressive state efforts to ensure that every child is well served, utilizing state assessments and leveraging accountability systems is a vital piece of effective school leadership. Public Agenda's 2003 survey of administrators asserts that 63 percent of superintendents report that raising student achievement is the biggest part of a principal's evaluation and that 47 percent have moved a successful principal to a low-performing school to help it improve (Farkas et al., 2003, p. 23). However, while 73 percent of today's superintendents also think holding principals accountable for student learning is a "good idea" and just 18 percent that it's a "bad idea," 45 percent of principals term the idea a "bad" one and just 41 percent a "good" one (Farkas et al., 2003, p. 38). Given that current principals express this kind of evident discomfort with accountability, how much attention are programs devoting to accountability in training principal candidates?

In practice, just 50 of 381, or about 13 percent, of "managing for results" course weeks linked school management to standards-based accountability systems, state assessments, or the new demands of No Child Left Behind. This means that only about 2 percent of all course weeks addressed accountability as it relates to school management. Accountability was mentioned fewer than five times in other contexts, such as those relating to school law or policy, but these instances only accounted for a fraction of the attention to the topic.

Table 3: Managing for Results with Accountability

Type of School Percentage of "Managing for Results" Course Weeks Linked to Accountability, State Assessments, or NCLB

Elite 13.3% (22 /166)

Large 14.0% (15 /107)

Typical 12.0% (13 /108)

Total 13.1% (50 /381)

Of course, as Theodore Creighton (2001) and others have noted, one precondition for using accountability as an effective management tool is that principals be equipped to make use of data, research, and the associated technology. How much attention are preparation programs devoting to these topics? Table 4 shows that the 381 course weeks devoted to "managing for results" involved "data, technology, or research" about 29 percent of the time. These course weeks addressed topics like "Data-driven decision-making . . . the process of data collection and analysis at the district level" (Syllabus 66, Week 7) and "The bottom line: The principal's impact on student achievement . . . [using] data analysis" (Syllabus 88, Week 15). There was much variation in how individual programs covered these topics.

Table 4: Managing for Results with Data and Research

Type of School Percentage of "Managing for Results" Course Weeks That Mention "Data, Technology, or Research"

Elite 28.3% (47/166)

Large 38.3% (41/107)

Typical 19.4% (21/108)

Total 28.6% (109/381)

Combining the results from Tables 3 and 4 reveals an estimate of the total amount of time devoted to using accountability, data, research, or technology as management tools. The bottom line is that perhaps 6–7 percent of instruction addressed one of these topics as it relates to managing school improvement. Course weeks that discussed managing for results without referencing accountability, data, or research often expressed a more spiritual approach to school improvement. One such course week asked, "How do we engage the moral and aesthetic imagination in the educational change process?" (Syllabus 57, Week 1).

Another way to characterize the attention paid to data and research is to ask what percentage of all course weeks included a description, reading, or assignment that referenced data or research. Overall, 263 out of 2,424, or 11 percent, of weeks referenced statistics, data, or empirical research in some context. In short, these results seem to reveal that issues of data and research receive very limited attention in principal-preparation programs.

Managing personnel

A critical role for any leader is hiring, evaluating, developing, and firing personnel. This category included course weeks addressing the hiring process (recruitment, selection, interviewing), employee motivation, teacher evaluation, and conflict management. About 15 percent of course weeks were devoted to personnel management.

While principals have always been limited in their ability to hire, remove, or reward personnel—due to state statutes, district procedures, staffing routines, collective bargaining provisions, and so on—they are now pressed both by expectations and by statute to play an increasingly aggressive role in ensuring teacher quality. In fact Education Week ("Leading for Learning", 2004) reports that 80 percent of principals believe they have a great deal of influence over evaluating teachers and that 74 percent say the same about hiring new personnel (p. S7).

In light of these developments, the question arises: How much are training programs doing to prepare principals for the challenges of personnel management? The importance of the issue is highlighted by a 1999 study of 54 U.S. companies which concluded that the cost of the average managerial "mis-hire" was 24 times the failed employee's starting salary (Smart, 1999). Table 5 illustrates that about 11 percent of the managing personnel course weeks addressed the various facets of the hiring process: recruitment, selection, interviewing, and placement. Coverage of the hiring process varied from program to program, with some paying no attention to recruitment, selection, or placement. In all, 21 of 31 programs, or 68 percent, covered the hiring process only once or not at all in all syllabi coded.

Table 5: Time Devoted to Recruitment and Hiring

School Type Percentage of "Managing Personnel" Course Weeks Addressing Hiring Process

Elite 7.0% (9/129)

Large 11.1% (14/126)

Typical 17.1% (18/105)

Total 11.4% (41/360)

While hiring good faculty is one challenge that principals face, a second critical task is evaluating personnel. How much time do various programs devote to helping principals learn to evaluate teachers? Table 6 shows that about 24 percent of the 360 course weeks that addressed managing personnel were devoted to teacher evaluation.

Table 6: Time Devoted to Evaluating Teachers

Type of School Percentage of "Managnng Personnel" Course Weeks Addressing "Teacher Evaluation"

Elite 31.8 % (41/129)

Large 20.6% (26/126)

Typical 17.1% (18/105)

Total 23.6% (85/360)

Table 6 suggests that a considerable percentage of personnel management is devoted to faculty evaluation. However, the obvious question that emerges is the degree to which principals are being equipped to make important decisions about tenure, compensation, and professional development based on those judgments.

Table 7 examines the focus of the 85 course weeks devoted to teacher evaluation. Weeks addressing teacher evaluation that encompassed the more agreeable, supportive elements of the process like "observation," "clinical supervision," "coaching," or "mentoring" were coded "supportive" evaluation. Meanwhile, weeks devoted to less genial topics, like linking evaluation to student performance, evaluating teachers using non-observational data, rewarding excellence, or "remediating" or dismissing low-performing faculty were coded "tough-minded" evaluation.

Table 7: Teacher Evaluation: Tough-Minded vs. Supportive

Type of School Teacher Evaluation Course Weeks Tough-Minded Supportive

Elite 41 26.8% (11/41) 73.2% (30/41)

Large 26 26.9% (7/26) 73.1% (19/26)

Typical 18 22.2% (4/18) 77.8% (14/18)

Total 85 25.9% (22/85) 74.1 % (63/85)

Table 7 shows that "tough-minded" evaluation received much less attention than did "supportive" evaluation. "Tough-minded" evaluation accounted for just 22 of 85 (26 percent) course weeks devoted to evaluation. This means that less than 1 percent of all instruction in the syllabi examined focused on aggressive efforts to identify, enhance, and reward teacher quality. Michael Fullan, one of the most frequently assigned authors in the syllabi, captured a common theme in the discussions of evaluation when he wrote, "Appraisal schemes that implicate 100 percent of the staff in order to detect a small percentage of incompetents are a gross waste of time . . . Any appraisal schemes should be decidedly focused on growth and development" (Fullan & Hargreaves, 1996, p. 10–11). In fact, course weeks routinely focused on procedural questions (e.g., "Cycles of supervision: What's due when," Syllabus 59, Week 7) or finding ways to support problematic staff (e.g., "Supervising the marginal teacher" Syllabus 135, Week 7; "Working with difficult people," Syllabus 188, Week 10).

To further examine this finding, we looked at how often managing personnel course weeks mentioned "tough-minded" notions like "termination," "dismissal," "firing," "teacher compensation," or "salary." Table 8 shows that just 3 percent of the personnel management course weeks made mention of teacher dismissal and less than 3 percent made mention of teacher compensation. In all, just 21 of 360 course weeks devoted to managing personnel addressed employee compensation or termination—two subjects that typically receive extensive attention when personnel management is addressed in other venues. Many programs did not discuss termination or compensation at all: 20 of 31 programs never mentioned termination, and 23 of 31 failed to mention compensation. None of the 31 programs mentioned either topic more than once.

Table 8: Dealing with Termination/Dismissal and Compensation

Type of School "Managing Personnel" Course Weeks Addressing Teacher Dismissal "Managing Personnel" Course Weeks Addressing Compensation

Elite 3.1% (4/129) 4.7% (6/129)

Large 5.6% (7/126) 0.8% (1/126)

Other 1.0% (1/105) 1.9% (2/105)

Total 3.3% (12/360) 2.5% (9/360)

This mindset contrasts sharply with the conventional wisdom in public- and private-sector management regarding the benefits of the selective use of evaluation, compensation, and termination for quality control. As former General Electric CEO Jack Welch (2001)—identified in a 2003 survey as one of the 50 "most important" living management thinkers—has explained: "Making these judgments is not easy, and they are not always precise . . . but . . .This is how great organizations are built. Year after year, differentiation raises the bar higher and higher and increases the overall caliber of the organization" (p. 158). Of course, the Welch model is not how schools currently operate, leaving open the question as to whether principals ought to be introduced to such thinking.

Overall, the preparation programs in the sample approach personnel management with little attention to teaching new principals to hire, evaluate, reward, or terminate employees. In fact, some syllabi evince a peculiar take on these issues, as in the case of the course week that archly made reference to, "The symbolism of attempting to fire an incompetent teacher" (Syllabus 41, Week 8).

Technical knowledge

While reformers discuss how to help principals focus more on improving school performance, the data suggest that principals continue to spend most of their work day on administrative issues and the nuts and bolts of running a school. For instance, a 2004 survey by Education Week revealed that 86 percent of principals report spending time each day on ensuring the security of their school and 82 percent spend part of their day managing facilities and resources ("Leading for Learning", 2004). Presumably due to this reality, technical knowledge was the most frequently addressed topic in the syllabi examined. It encompassed 30 percent of the total course weeks—nearly twice the time devoted to any of the other six content areas.

The "technical knowledge" category included those course weeks which focused on topics like school law, school finance, facilities management, data training, research, technology, and related concerns like meeting coordination and food and transportation services. Data training and research were coded as technical knowledge when they were presented in the abstract rather than in a context linking them to schooling— in those weeks where use of data and research were linked to topics like assessment or program evaluation, "research" and "data" were coded as "managing for results" and included in Table 4.

What topics do preparation programs focus upon when teaching technical knowledge? Table 9 shows that nearly three-quarters of the time devoted to technical knowledge was spent on education law and school finance. Of 718 course weeks devoted to technical knowledge, 324 (about 45%) focused on education law, and 196 (27%) on school finance. The remaining course weeks addressed research skills (10%), data management and utilization (7%), technology (5%), and facilities (4%). There was some variation across different kinds of programs. Elite programs devoted more than 85 percent of their attention to law and finance but just 5 percent to data, research skills, and technology. Meanwhile, both large and typical programs spent 27–32 percent of their technical knowledge instruction on data, technology, and research skills and 62–68 percent of their time on finance and law.

Table 9: The Coverage of Technical Leadership Skills

Type of School Finance Law Facilities Data training Research

Skills Technology Other

Elite 37.6%

(91/242) 50.8%

(123/242) 4.1%

(10/242) 4.1%

(10/242) 1.7%

(4/242) 0%

(0/242) 1.7%

(4/242)

Large 16.1%

(48/298) 45.6%

(136/298) 4.7%

(14/298) 10.7%

(32/298) 14.4%

(43/298) 7.0%

(21/298) 1.3%

(4/298)

Typical 32.0%

(57/178) 36.5%

(65/178) 3.9%

(7/178) 6.2%

(11/178) 14.0%

(25/178) 7.3%

(13/178) 0%

(0/178)

Total 27.3%

(196/718) 45.1%

(324/718) 4.3%

(31/718) 7.4%

(53/718) 10.0%

(72/718) 4.7%

(34/718) 1.1%

(8/718)

The titles of the technical knowledge course weeks raise some questions about the body of skills regarded as essential to school operations. Some courses on law, finance, facilities, and so on seem to have straightforward value, as with the weeks titled, "Preparation and interpretation of financial statements…[including] managing the accounting and control function" (Syllabus 19, Week 4) or "Revisit[ing] basic data distributions…[including] percentiles, inter-quartile ranges, descriptive statistics" (Syllabus 78, Week 2). On the other hand, some syllabi appear less focused on useful expertise, particularly those with lessons on "Financial equity: Should all students have a constitutional right to an equal education? Should this require an equal or equitable allocation of funds to all schools?" (Syllabus 20, Week 9) or "Topics of silent, political, passive speech . . . valuing student speech, supporting versus tolerating speech . . . and civic and civil development of students" (Syllabus 51, Week 4). On the whole, however, a plain reading of the syllabi reveals that more than 85 percent of technical knowledge course weeks focused on operational and applied tasks.

Because topics like law, facilities, finance, and technology cover facts and formulas that principals are thought to need, we examined the extent to which courses teaching "technical knowledge" use final assessments to ensure mastery. We found that about 58 percent of 64 "technical knowledge" courses required students to take a final exam. Interestingly, while two-thirds of large and typical program courses in technical knowledge required exams, just 39 percent of the 23 elite courses examined did.

External leadership

We also considered how much instruction was devoted to "external leadership" responsibilities such as working with constituencies like parents and community organizations, negotiating local politics, understanding collective bargaining agreements, and developing small business skills like marketing and public relations. In the aggregate, external leadership was addressed in about 8 percent of course weeks.

Table 10: Allocation of Time Within "External Leadership"

School Type School Board Parental Relations Small Business Skills Community Relations Collective Bargaining Politics/Policy Other

Elite 4.3%

(4/92) 4.3%

(4/92) 6.5%

(6/92) 29.3%

(27/92) 13.0%

(12/92) 33.7%

(31/92) 8.7%

(8/92)

Large 6.8%

(4/59) 0%

(0/59) 23.7%

(14/59) 20.3%

(12/59) 22.0%

(13/59) 22.0%

(13/59) 5.1%

(3/59)

Typical 2.3%

(1/43) 2.3%

(1/43) 16.3%

(7/43) 11.6%

(5/43) 2.3%

(1/43) 34.9%

(15/43) 30.2%

(13/43)

Total 4.6%

(9/194) 2.6%

(5/194) 13.9%

(27/194) 22.7%

(44/194) 13.4%

(26/194) 30.4%

(59/194) 12.4%

(24/194

Within external leadership, Table 10 shows how much attention was devoted to specific topics of interest. The most attention was devoted to understanding politics and policy (30 percent of course weeks) and to community relations (23 percent). Small business skills and collective bargaining received a great deal of attention in large programs, accounting for almost half of their external leadership course weeks, but only a fraction of that in elite or typical programs. Parental relations and dealing with the school board received minimal attention across the board, together accounting for only about 7 percent of external leadership course weeks.

Norms and values

Education school critics assert that courses are often characterized by an ideological tilt that influences content (Steiner & Rozen, 2004). Does the evidence suggest such a bias in the case of principal-preparation courses? The data in Table 2 suggests that only about 12 percent of course weeks explicitly focus on promoting particular norms or values. In the instruction that does address norms and values, is there evidence to support the claim that instructors are failing to expose students to diverse points of view?

Obviously, gauging the ideological content inevitably requires the researcher to make judgment calls.7 All we can do here is assuage concerns by explaining the coding rules and providing examples to show how the coding metric was applied. Coded as "progressive" were course weeks that advocated concepts like social justice and multiculturalism, focused on inequality and race-based discrimination, emphasized notions of silenced voices and child-centered instruction, or were critical of testing and choice-based reform. Those weeks that critiqued notions of social justice and multiculturalism, critiqued a focus on inequality or discrimination as engaging in "victimhood," advocated phonics and back-to-basics instruction, or framed discussions of testing or choice-based reform in positive terms were coded as "conservative." Those weeks that did not display clear normative direction or that included a variety of normative views were coded "neutral."

For examples of how this metric was applied, some of the course weeks that were coded as progressive included:

• "The role of the curriculum in legitimating social inequality" (Syllabus 8, Week 10)

• "What role(s) do race and social class play in school reform? Is social Darwinism a useful reform concept?" (Syllabus 35, Week 9)

• "Other silenced voices? (females, gay, impaired, over/underweight, bullied, biracial, religion, homeless, transient, etc.). Sexism and gender roles? Equity? Heterosexism and gay issues?" (Syllabus 40, Week 11)8

• Course weeks that were coded as balanced/neutral/ or indeterminate included:

• "Are unions good or bad for public education? What does the evidence say?" (Syllabus 22, Week 2)

• "Educational productivity, assessing educational productivity, does money matter? Improving productivity . . . class size, vouchers, charter schools" (Syllabus 33, Weeks 8 & 9)

• "What should schools teach? Phonics vs. whole language; multicultural education/teaching for diversity" (Syllabus 161, Week 3).

The single course week coded "conservative" was titled "The state and local politics of education reform" (Syllabus 23, Week 12). It was coded as such because the week's primary reading was authored by a well-known scholar regarded as conservative.

Table 11 shows the distribution of normative perspectives during the "norms and values" course weeks.

Table 11: Distribution of Normative Perspectives

Type of School Norms and Values Course Weeks Progressive Conservative Balanced/Neutral/

Indeterminate

Elite 145 70.3%

(102/145) 0.6%

(1/145) 29.7%

(42/145)

Large 67 65.7%

(44/67) 0.0%

(0/67) 34.3%

(23/67)

Typical 81 54.3%

(44/81) 0.0%

(0/81) 45.7%

(37/81)

Total 293 64.8%

(190/293) 0.3%

(1/293) 34.8%

(102/293)

Table 11 suggests that there is a distinct progressive tilt in those weeks that address norms and values. Overall 65 percent of the norms and values weeks were coded as progressive, 35 percent as balanced, neutral, or indeterminate, and less than 1 percent as conservative. Contrary to earlier suggestions that elite programs were atypically biased, the progressive tendency was evident at all three kinds of programs. It was most marked at elite and large programs, but even at typical programs, the majority of course weeks were progressive and none were conservative. Interestingly, many of the issues of political correctness traditionally attacked by education school critics were not much in evidence. For instance, the words "diversity" and "diverse" and "multiculturalism" and "multicultural" appeared in only 3 percent of all course weeks.

In the end, however, the imbalance of ideological perspectives does raise cautionary flags about instructors' interest in entertaining competing schools of thought on leadership. For instance, courses titled "Leadership, diversity, and social justice" (Syllabus 174) or weeks labeled "Suturing together a conservative public agenda: markets, religion, standards, and inequality" (Syllabus 8, Week 9) raise doubts about whether the aim is to educate or to promote a particular point of view. Still, it is important to note that these principal-preparation programs only devote slightly more than 10 percent of instructional time to norms and values.

Most frequently read authors

One way to delve deeper into the content of instruction is to examine which authors are included on course reading lists in principal preparation programs. Answering these questions required a way to systematically assess assigned readings. Only "required readings" were tallied; "recommended readings," "supplementary readings," and "suggested readings" were excluded. Authorship was coded as follows: A reading was attributed to an individual if they were first or second author. In cases where there were more than two authors, if the scholar in question was not first or second author, the reading was not attributed to that author. Edited volumes themselves are not attributed, although chapters assigned within edited collections—including introductions or conclusions authored by the editor—are attributed to the author and tallied. The most commonly assigned authors are listed in Table 12.

Table 12: Most Commonly Assigned Authors

Author Number of Times Assigned

Terrence Deal 25

Allan Odden 25

Kent Peterson 24

Michael Fullan 15

Lee Bolman 14

Thomas Sergiovanni 13

David Schimmel 13

Richard Elmore 13

Linda Darling-Hammond 12

Wayne Hoy 11

Deborah Meier 9

N: 1,851 readings

The most commonly assigned authors were Terence Deal (University of Southern California) and Allan Odden (University of Wisconsin). Deal is a prolific scholar of school leadership; Odden a scholar of education finance, state funding systems, and related issues. Other commonly assigned authors were Kent Peterson (University of Wisconsin); Michael Fullan (University of Toronto), Lee Bolman (University of Missouri-Kansas City), Thomas Sergiovanni (Trinity University, TX), Richard Elmore (Harvard Graduate School of Education), Linda Darling-Hammond (Stanford University), and Deborah Meier (founder of Central Park East school). These authors tend to focus on the "unique" challenges of school leadership. Sergiovanni (1996), for instance, argues that preparation for school leadership is unlike that for other leadership roles, declaring that "corporate" models of leadership cannot work in education and that, "We [must] accept the reality that leadership for the schoolhouse should be different, and . . . we [need to] begin to invent our own practice" (p. xiv).

It is worth noting that several of these thinkers—particularly Darling-Hammond, Meier, Sergiovanni, and Fullan—are unapologetic skeptics of test-based accountability systems, standardized assessments, pay-for-performance, competitive pressure, and other key elements of the new educational environment.9 While some of the authors are more balanced in their stance on such measures, nowhere on this list are there identifiable proponents of test-based accountability, expedited termination of poor teachers, educational competition, or related measures that are changing the managerial context.

To determine whether this impression may be an artifact of the list of most commonly read authors, Table 13 lists a number of influential education thinkers and how often their writings were assigned. While specialists in educational administration were frequently assigned, largely absent were influential scholars of education governance and productivity like Paul Hill, Larry Cuban, William Boyd, Michael Kirst, and Jim Guthrie.

Table 13: The Representation of 10 Key Education Governance/Productivity Thinkers

Author Number of Times Assigned

Eric Hanushek 6

Henry Levin 4

Richard Murnane 2

James Guthrie 2

Larry Cuban 2

Chester E. Finn, Jr. 2

Michael Kirst 1

William Boyd 1

Herbert Walberg 0

Paul Hill 0

N: 1,851 readings

Table 14 addresses the question of how frequently leading management thinkers were read in the courses studied. A 2003 survey of business leaders, business school professors, and MBA students by Bloomsbury Publishing and Suntop Media produced a list of the 50 "most important" living management thinkers (Thinkers 50, 2003). Of the individuals identified by the Thinkers 50 rankings, Table 13 shows that only nine were assigned at all, and they were assigned a combined total of just 29 times in all of the courses studied.

Table 14: Frequency of Thinkers 50 Management Theorists

Author Number of Times Assigned

Peter Senge 7

Stephen Covey 4

Edgar Schein 4

Rosabeth Moss Kanter 3

Peter Drucker 3

John Kotter 3

Henry Mintzberg 2

Warren Bennis 2

Daniel Goleman 1

N: 1,851 readings

In other words, of the 1,851 readings contained in the sample, a total of just 1.6 percent were authored by one of the 50 thinkers deemed most influential by management students, teachers, and practitioners. Influential thinkers including Michael Porter, Jim Collins, Clay Christensen, and Tom Peters—some of whom are widely cited in education circles and asked to address national educational conferences—were wholly absent from every syllabus in the sample. The omission of these scholars, along with the inattention to serious education management thinkers, raises questions about whether principals are adequately steeped in important thinking and research on management and productivity.

For instance, when addressing school culture and climate, the most commonly assigned readings tend to focus almost exclusively on the importance of creating warm, supportive, and nurturing school environments. One of the most commonly assigned texts on school culture declares, "The social climate and culture of a school influence the emotional and psychological orientation of its staff. . . . This is especially the case in schools that are optimistic, socially caring and supportive, and energetic" ( Peterson & Deal, 1999, p. 8). Offering a rather different take, Jim Collins, one of the Thinkers 50 who appears nowhere among the assigned readings, has argued that, "Visionary companies are not exactly comfortable places. . . visionary companies thrive on discontent. They understand that contentment leads to complacency, which inevitably leads to decline" (Collins & Porras, 1997, p. 186–87). The point is that both sets of authors offer valuable, even crucial, insights, but that one school of thought appears to be largely absent from the reading lists.

While educational leadership lies at the intersection of two vibrant and powerful bodies of learning and thought—education and leadership/management—instruction in the programs examined draws narrowly from a pool of "educational administration" specialists. This is particularly problematic because growth and advancement come not merely from new inventions or unearthing new resources but from entrepreneurial leaders and decision-makers configuring personnel, practices, operations, and existing resources in new and more effective ways. Of course, educational leaders will be far better equipped to do so if they are familiar with a broad body of knowledge on learning, technology, management, and productivity.

CONCLUSION

This study presents a comprehensive assessment of what aspiring principals are taught in a national sample of 31 preparation programs and reveals considerable consistency across a variety of institutions. Our research raises questions about whether preparation is well matched to the contemporary world of schooling and whether graduates of principal-preparation programs are being equipped for the challenges and opportunities posed by an era of accountability. As three professors of educational leadership have observed, "Leadership during this Age of Accountability has become more stressful, more political, more complex, and more time-consuming" but that school leaders have also been given unprecedented "clarity of mission" and "leverage to bring teachers into line" (Duke, Grogan, & Tucker, 2003, p. 212). The recent study by Arthur Levine (2005), President of Teachers College at Columbia, posed thoughtful structural remedies intended to address these needs. Among Levine's suggestions were to create an education management degree like the MBA, to eliminate the Ed.D, and to stop districts from offering pay raises for course credit. Such structural changes are certainly deserving of careful scrutiny, but Levine's study also raises a more fundamental question as to whether the content of preparation courses may also need to be rethought.

Ultimately, the question of instructional content is pivotal; yet we find that principals currently receive limited training in the use of data, research, technology, the hiring or termination of personnel, or evaluating personnel in a systematic way. The reading lists suggest that aspiring principals receive limited exposure to important management scholarship or sophisticated inquiry on educational productivity and governance. The vital question is whether the lack of attention to certain schools of thought regarding management may leave aspiring principals prepared for the traditional world of educational leadership but not for the challenges they will face in the 21st century. Principal-preparation programs that pay little attention to data, productivity, accountability, or working with parents may leave their graduates unprepared for new responsibilities. This system risks graduating new principals unprepared to exercise new freedoms or responsibilities—resulting in micromanagement, poor decisions, or the misuse of accountability instruments. Meaningful reform of principal-preparation programs must ensure that the content of these programs it is well suited for the challenges confronting principals in a new era of schooling. While these data cannot offer any definitive guidance with regard to these important questions, the queries are critical ones that demand more careful exploration.

This study represents an initial cut at understanding how aspiring principals are being prepared. Our work is intended as a provocative first look—not as the final word. Our hope is that scholars of educational administration and school leadership will refine, extend, and challenge our inquiry and analysis. In light of the numerous efforts to rethink principal preparation currently under way across the country, we hope that researchers will document and monitor the changes wrought by ongoing efforts to reform preparation. In particular, given the exploratory nature of this study, it will be appropriate for future research employing more comprehensive samples to assess the relative magnitude of the differences we find across program categories and within areas of instruction.

An important question raised by syllabi studies such as this is the degree to which syllabi actually reflect what takes place inside classrooms. While syllabi can be imperfect indicators of classroom instruction, whether they can serve as valuable proxies for what concepts are being taught is an open question. We hope that future inquiries will help to make clear how closely principal-preparation courses resemble the syllabi on which they are based. More importantly, such research could help shed light on what gets taught in principal-preparation courses. For instance, it would be most interesting to learn what instructional approaches are used, what topics are discussed, what kind of work is assigned, and how is performance assessed. Future research might also explore the distinction between what principals, assistant principals, and specialists need to know and how those differences are recognized or addressed in the course of preparation.

Of course, the content of instruction is only one component of more fully understanding the nature of principal preparation; it will also be equally useful for scholars to consider internships, program structure, and mentoring. Our hope is that scholarship on preparation will examine both program structure and content.

Currently, a number of preparation programs are considering redesign. This means that these findings may become dated as new training programs are launched in the years ahead. It also means that these findings could be of immediate value in helping principal-preparation providers to revisit their assumptions as they work to rethink their programs. That said, the Southern Regional Education Board, whose Leadership Initiative is driving preparation and certification reform in its 16 member states, has cautioned: "Redesigning leadership-preparation programs does not mean simply rearranging old courses—as staff at some universities and leadership academies are inclined to do. True redesign requires a new curriculum framework and new courses aimed at producing principals who can lead schools to excellence" (SREB, 2003, p. 7). Whether preparation programs are intent on answering that challenge remains to be seen, but the flurry of redesign efforts is a development worth watching in the years ahead.

Notes

1 Jamentz (2002) highlights this set of skills and asserts that principals must be actively engaged in constructing standards-based curricula, aligned assessments, and demonstrating and coaching effective teaching and learning practices.

2 While scholars have examined the link between the values and beliefs of administrators and effective schooling (Krug, Ahadi, & Scott, 1991), looked at "clusters" of leadership behavior and their relationship to school performance (Heck, Larson, & Marcoulides, 1990), and reviewed previous research to determine the importance of the principal's role in effective schooling (Hallinger & Heck, 1998), this work is regarded as more suggestive than conclusive.

3 Other studies analyzed the effect of extra training in "interpersonal management" on candidates' interpersonal skills like "physical attending, empathy, respect, and concreteness" (Smith, Montello, & White, 1992, p. 243) and surveyed administrators to identify skills that they report principals need (Schmeider, McGrevin, & Townley, 1994; Daresh, Gantner, Dunlap, & Hvizdak, 2000).

4 It is worth noting that principal-preparation programs do not solely serve aspiring principals. The classes in these programs also enroll assistant principal candidates, specialists (like special education administrators or school finance officials), and teachers seeking graduate credit. However, this fact should not confuse our line of inquiry. Since we sampled syllabi from core courses in each institution's principal-preparation track, the fact that these courses may enroll others does not alter the fact that they are the classes that future principals complete during their graduate preparation.

5 Because principal candidates will often take classes at two or more different institutions over the course of obtaining their credential, schools often do not keep records of the number of principals certified each year and the IPEDS database does not keep track of the number of certificates that are awarded by individual schools.

6 Three of the 210 syllabi included in the final sample did not have specific topics listed but did include assigned readings. Because the topic of the session could be gleaned from the readings, these syllabi were included.

7 It is, of course, theoretically possible for faculty to assign readings of one ideological perspective and then use instructional time to ensure balanced consideration. This analysis, however, proceeds on the premise that class instruction and course readings are important determinants of what students learn. We presume that courses in which students are exposed to a variety of perspectives are more likely to foster open inquiry and instruction.

8 Other examples include, "Transforming urban education with a social justice agenda: The role of the Brown decision in the fight for social justice in urban communities" (Syllabus 92: Week 2); Fomenting social change" (Syllabus 128: Week 6); and "Tools for change: Social reconstructionist schooling" (Syllabus 174: Week 4).

9 For instance, Linda Darling-Hammond and Deborah Meier are among the contributors to a recent volume that assails No Child Left Behind-style accountability entitled Many Children Left Behind: How the No Child Left Behind Act is Damaging Our Children and Our Schools (3Meier & Wood, 2004).

References

Bagin, D., & Gallagher, D. R. (2001). The school and community relations. 7th edition. New York: Allyn and Bacon.

Bozeman, W. C., & Spuck, D. (1991). Technological competence: Training educational leaders. Journal of Research on Computing in Education, 23(4), 514.

Butin, D. W. (2004). The foundations of preparing teachers: Are education schools really "intellectually barren" and ideological? Teachers College Record July 24.

Cochran-Smith, M. (2004). Walking the road: Race, diversity, and social justice in teacher education. New York: Teachers College Press.

Collins, J. C., & Porras, J. I. (1997). Built to last: Successful habits of visionary companies. New York: Harper Collins.

Creighton, T. (2001). Data analysis and the principalship. Principal Leadership, 1(9), 52–57.

Daresh, J. C., Gantner, M., Dunlap, K., & Hvizdak, M. (2000). Words from "The Trenches": Principals' perspectives on effective school leadership characteristics. Journal of School Leadership, 10(1), 69–83.

DuFour, R. (2002) The learning-centered principal. Educational Leadership, 59(8), 12–15.

Duke, D., Grogan, M., & Tucker, P. (2003). Educational leadership in an age of accountability. In D. Duke, M. Grogan, P. Tucker, & W. Heinecke, (Eds.), Educational Leadership in an Age of Accountability. New York: State University of New York Press.

Leading for Learning. (2004). Education Week, 24(3), S1–S7.

Elmore, R. (2000). Building a new structure for school leadership. New York: The Albert Shanker Institute.

Farkas, S., Johnson, J., & Duffett, A. (2003) Rolling up their sleeves: Superintendents and principals talk about what's needed to fix public schools. New York: Public Agenda.

Ferrandino, V., & Tirozzi, G. N. (2004). School leadership's future. Principal's perspective. Reston, VA: National Association of Secondary School Principals. Retrieved on September 12, 2006, at www.principals.org/s_nassp/bin.asp?TrackID=&SID=1&DID=46536&CID=34&VID=2&DOC=FILE.PDF

Fullan, M., & Hargreaves, A. (1996). What's worth fighting for in your school? New York: Teachers College Press.

Garland, V. E. (1990). Planning for the 1990's: Computer use in programs preparing school administrators. Planning and Changing, 20(4), 231–236.

Glasman, N., Cibulka, J., & Ashby, D. (2002). Program self-evaluation for continuous improvement. Educational Administration Quarterly, 38(2), 257–288.

Gonzalez, M., Glasman, N., & Glasman, L. (2002). Daring to link principal preparation programs to student achievement in schools. Leadership and Policy in Schools, 1(3), 265–283.

Grogan, M., & Andrews, R. (2002). Defining preparation and professional development for the future. Educational Administration Quarterly, 38(2), 233–256.

Hallinger, P., & Heck, R. (1998). Exploring the principal's contribution to school effectiveness: 1980–1995. School effectiveness and school improvement, 9(2), 157–191.

Heck, R., Larson, T. J., & Marcoulides, G. (1990). Principal instructional leadership and school achievement: Validation of a causal model. Educational Administration Quarterly, 26(2), 94–125.

Herrity, V. A., & Glasman, N. S. (1999). Training administrators for culturally and linguistically diverse school populations: Opinions of expert practitioners. Journal of School Leadership, 9(4), 235–253.

Hess, F. M. (2003). A license to lead? A new leadership agenda for America's schools. Washington, DC: Progressive Policy Institute.

Hess, F. M. (2004). Treating principals like leaders. American School Board Journal, 191(5), 32–35.

Hirth, M. A., & Valesky, T. C. (1990). Survey of universities: Special education knowledge requirements in school administrator preparation programs. Planning and Changing, 21(3), 165–172.

Hoy, W. K., & Miskel, C. (2005). Educational administration: Theory, research, and practice, 7th Edition. New York: McGraw Hill.

Jackson, B., & Kelley, C. (2002). Exceptional and innovative programs in educational leadership. Educational Administration Quarterly, 38(2), 192–212.

Jamentz, K. (2002). Isolation is the enemy of improvement: Instructional leadership to support standards-based practice. San Francisco: WestEd.

Keller, B. (2003). Education school courses faulted as intellectually thin. Education Week, 23(11), 8.

King, D. (2002). The changing shape of leadership. Educational Leadership,59(8), 61–63.

Krug, S., Ahadi, S., & Scott, C. K. (1991). Current issues and research findings in the study of school leadership. In P. Thurston & P. Zodhiates (Eds.), Advances in educational administration, Volume 2. Greenwich, CT: JAI Press.

Kowalski, T. (1995). Keepers of the flame: Contemporary urban superintendents. Thousand Oaks, CA: Corwin.

Kowalski, T. (2004). The ongoing war for the soul of school administration. Better leaders for America's schools: Perspectives on the manifesto (pp. 92–114). Columbia, MO: University Council for Educational Administration.

Lampe, R. E. (1985). Principals' training in counseling and development: A national survey. Counselor Education and Supervision, 25(1), 44–47.

Lashway, L. (2003). Transforming principal preparation. Washington DC: U.S. Department of Education, Office of Educational Research and Improvement.

Lashway, L. (2002). Developing instructional leaders. ERIC Digest, 160 (July).

Levine, A. (2005). Educating school leaders. New York: Teachers College, The Education Schools Project.

Lyman, L., & Villani, C. (2002). The complexion of poverty: A missing component of educational leadership programs. Journal of School Leadership, 11(4), 246–280.

Meier, D., & Wood, G. (2004). Many children left behind: How the No Child Left Behind Act is damaging our children and our schools. Boston, MA: Beacon Press.

Meier, D. (1995). The power of their ideas, lessons to America from a small school in Harlem. Boston, MA: Beacon Press.

Murphy, J. (2001a). The changing face of leadership preparation. School Administrator, 58(10), Web edition.

Murphy, J. (2001b). Reculturing the Profession of Educational Leadership: New Blueprints. Paper presented at the annual conference of the University Council for Educational Administration, November 3, Cincinnati, OH.

Murphy, J., & Vriesenga, M. (2004). Research in preparation programs in educational administration: An analysis. Monograph prepared for the University Council for Educational Administration.

Nicolaides, N., & Gaynor, A. K. (1992). The knowledge base informing the teaching of administrative and organizational theory in UCEA universities: A descriptive and interpretive survey. Educational Administration Quarterly, 28(2), 237–265.

Norton, M. S., & Levan, F. (1987). Doctoral studies in educational administration programs in UCEA member institutions. Educational Considerations, 13(1), 21–24.

Parker, L., & Shapiro, J. P. (1992). What is the discussion of diversity in educational administration programs? Graduate students' voices addressing an omission in their preparation. Journal of School Leadership, 2(1), 7–33.

Peterson, K., & Deal, T. (1999). Shaping school culture: The heart of leadership. San Francisco: Jossey-Bass.

Portin, B., Schneider, P., DeArmond, M., & Grundlach, L. (2003). Making sense of leading schools: A study of the principalship. Seattle, WA: Center on Reinventing Public Education.

Rebore, R.W. (2004). Human resources administration in education: A management approach, 7th Edition. New York: Pearson-Allyn and Bacon.

Roberts, J. (1991). In preparation for the principalship: Initiates' problems in conducting instructional conferences. Journal of Educational Administration, 29(2), 38–49.

Ryan, K., & Bohlin, K. E. (1999). Building character in schools. San Francisco: Jossey-Bass.

Schlechty, P. C. (2000). Leading a school system through change: Key steps for moving reform forward. In M. Fullan (Ed.), The Jossey-Bass Reader on Educational Leadership. San Francisco: Jossey-Bass. 182–201.

Schmeider, J. H., McGrevin, C., & Townley, A. H. (1994). Keys to success: Critical skills for novice principals. Journal of School Leadership, 4(3), 272–293.

Sergiovanni, T. (1992). Moral leadership. San Francisco: Jossey-Bass.

Sergiovanni, T. (1996). Leadership for the schoolhouse. San Francisco: Jossey-Bass.

Sirotnik, K A., & Kimball, K. (1994). The unspecial place of special education in programs that prepare school administrators. Journal of Educational Administration, 4(6), 598–630.

Smart, B. D. (1999). Topgrading: How leading companies win by hiring, coaching, and keeping the best people. New York: Prentice Hall.

Smith, R. M., Montello, P. A., & White, P. E. (1992) Investigation of interpersonal management training for educational administrators. Review of Educational Research, 85(4), 242–245.

Southern Regional Education Board (SREB). (2003). Good principals are the key to successful schools: Six strategies to prepare more good principals. Atlanta, GA: Southern Regional Education Board.

Steiner, D. M., & Rozen, S. (2004). Preparing tomorrow's teachers: An analysis of syllabi from a sample of America's schools of education. In F. M. Hess, A. J. Rotherham, & K. Walsh (Eds.), A qualified teacher in every classroom? Appraising old answers and new ideas (pp. 119–148). Cambridge: Harvard Education Press

Suntop Media and Bloomsbury Publishing. (2003). Thinkers 50 2003. London: Bloomsbury Publishing. Retrieved on September 1, 2004 from http://www.thinkers50.com/.

Supovitz, J. A., & Poglinco, S. M., (2001). Instructional leadership in a standards-based reform. Philadelphia, PA: Consortium for Policy Research in Education.

Sweeney, J., & Moeller, L. (2001). "Decision training – The use of a decision curriculum with in basket simulation. Education, 104(4), 414–418.

Thomas B. Fordham Foundation. (2003). Better leaders for America's schools: A manifesto. Washington, DC: Thomas B. Fordham Foundation.

Tucker, M. (2003). Out With the Old. Education Next, 3(4), 20–24.

Tucker, M., & Codding, J. B. (Eds.) (2002). The principal challenge: Leading and managing schools in an era of accountability. San Francisco: Jossey-Bass.

Welch, J. (2001). Jack: Straight from the gut. New York: Warner Books.

Young, M., & Kochan, F. (2004). UCEA leaders respond: Supporting leadership for America's schools. Better Leaders for America's Schools: Perspectives on the Manifesto (pp. 115–129). Columbia, MO: University Council for Educational Administration.

Young, M., Peterson, G., & Short, P. (2002). The complexity of substantive reform: A call for interdependence among key stakeholders. Educational Administration Quarterly, 38(2), 130–136.

Cite This Article as: Teachers College Record Volume 109 Number 1, 2007, p. -

http://www.tcrecord.org ID Number: 12742, Date Accessed: 12/20/2006 3:48:20 PM

by Robert McClintock — March 28, 2007

Educational research accumulates in great, growing bulk, with all manner of contradictory findings, and no leverage by which to effect practice in any significant way. Better schooling depends, less on research, but on adequate resources for the job, human and financial, and lots of hard work, day by day, in an ethos of support and high expectation, in school and out.

Don't take it personally—I'm sure your research is great, but taken all together, educational research has become absurd, out of harmony with sound judgment. Unfortunately, we have come to know too well that absurdity in the seat of power causes human harm.

Consider first the absurdity. Educational research has fattened horribly. Yes, that may offend, but sometimes a friend must be honest. Researchers don't notice the excess as they worry about too much here or too little there, over-extending the quantitative, too little qualitative, not enough of the triangulated, the doubly blinded. They rush too quickly after the method du jour, hoping to chomp data into another published roll of flab, by which the oleaginous field oozes up the ladder of promotion and tenure.

A gross exaggeration? AERA is accepting proposals for new handbooks of educational research and to get an inkling of what to expect, look closely, for instance, wondering with skeptical admiration, at the Handbook of Research on Teaching, since 2001 in its fourth edition. As AERA suggests, this "resource for students and scholars in education and beyond…will inform practice-policy, school administration, teaching, instruction, and parenting." Is it fit for the effort?

At $85, the member price, the Handbook offers value, but $15 for shipping tips us off. It has heft – circa 7 pounds, just under 1,300 pages, 8.5 by 11, double columns, set in 10 point Times Roman, ungenerous margins. Eighty-five distinguished authorities contribute 51 articles, each on average discussing 160 research studies. The subject index identifies over 4,000 topics of note; the name index cites 7,130 persons who merit attention. The Handbook covers the field, of course. Multiple articles on the intellectual foundations and the methodologies of research open it, and surveys of research on teaching key subjects, on the learner, on policy, on teachers and teaching, on the social and cultural contexts of teaching, and finally on the practice of instruction all follow. Indeed, the Handbook is an epitome of research on teaching, the concern at the heart of educational research writ large.

Does the Handbook represent an absurd effort? It epitomizes thousands of person-years of methodical inquiry, each piece of it fine fare, but in bulk indigestible. It assembles work, pointless in a deep, existential sense, for the research goes off in every direction, leaving those in "practice-policy, school administration, teaching, instruction, and parenting" without a clue what to do. Before them, a research landscape spreads out, a vast plain, with a hillock here and there among the dead (Dewey, Freire, Piaget, and Vygotsky) and the living (Shulman, Darling-Hammond, and a few others). The contributors cite their 7,000 plus researchers on average a little over twice each and mention the median researcher only 3 times. What is the message? How should the eager reader react to the 4,200 isolated studies, which the Handbook mentions only once in its million or so words? Which of them is the key unlocking whatever problem is at the reader's hand?

Can the policy maker at the federal, state, or local level use the Handbook, or the thousands of studies to which it points? Perhaps, for this or that, but in a helter-skelter manner. Will the Handbook help the harried school administrator, pressed to meet standards, eager for his school to shine on tests, struggling to keep things humming on a parsimonious budget, having to cope with the kids living at the margin and testing every boundary? And can those engaged in teaching and instruction work with the Handbook, and all the research it represents, as they engage daily 120 protean persons, each distinctly different, in the trial of mastering some skills, values, and knowledge? With the Handbook in hand, follow a teacher, good or bad, for a week, and try to offer sage advice from it as the teacher sees teachable moments flit forth for this pupil and that one. And then, can someone parenting use the Handbook well? Imagine him or her after a long day at work, settling into bed, 7 pounds of book resting on stomach muscles that have begun to sag from fatigue and age, intently squinting at the small-print on double-columned pages, and on 919 exclaiming suddenly, "Honey, here's the answer!"

In the lexicon of educational researchers, "education" denotes schooling and researchers must start asking to what degree their work can determine the realities of schooling. These realities are imperious, domineering, imperative. Schools work the same way the world around; they have powerful routines and massive institutional inertia. Tradition, ritual, convention, interest, bureaucratic procedure, folly, the lore of practice, and expedient intelligence determine the actualities of schools.

Let's be honest and humble: because educational researchers have proven unable to exercise rigorous control and account for the relevant variables in carefully designed inquiries, their studies have had notoriously conflicting results. If researchers cannot master the variables in controlled settings, why expect practitioners, caught in institutional cross-currents and daily coping with complexity, to be able to rationalize school activities according to the prescripts of research findings? They can't; they won't; the direct application of research will have no coherent effects.

Oh, yes—perhaps direct application has not been the point. Educational researchers also staff the professional schools of education as the faculty members who educate the educators. Perhaps as the researchers prepare pre-service novices and bring in-service practitioners up-to-date with newly proven techniques, their educational research will shape practice indirectly. This faith constitutes the educational version of the great trickle-down delusion. Do we believe the very rich when they assure us that their getting ever richer will soon improve all our pay, benefits, and security? Alas, the trickling down of educational research from the schools of education will have even less effect than do the leavings of great wealth upon the annual income of the ordinary person.

So, in sum, the absurdity: educational research accumulates in great, growing bulk, with all manner of contradictory findings, and no leverage by which to effect practice in any significant way. Better schooling depends, less on research, but on adequate resources for the job, human and financial, and lots of hard work, day by day, in an ethos of support and high expectation, in school and out.

Consider now the harm. The vast quantity of educational research produced year in, year out, serves no real need or opportunity in the workaday world of schools, of their management, or of parenting. It does not arise to meet a felt demand from these quarters. It exists because the system of schooling requires many teachers and they require a professional preparation, which occurs primarily in academic institutions. In turn, those academic institutions need a faculty and they assess the ability to conduct and publish research as their primary criterion for deciding who to recruit, promote, and tenure as faculty members. The vast bulk of educational research will have no effect on anything except the process of recruitment, promotion, and tenure in schools of education. It exists for the sole reason that both individual researchers and the institutions that employ them consistently use the research for this purpose.

This assertion will come as no surprise to anyone who has spent much time with eyes open in schools of education. But what harm does the practice do? Four injuries result.

• Skill in educational research often has little to do with the real educational work that should take place in the professional preparation of educators. By relying on the demonstration of extraneous research skills as the prime criterion for recruitment, promotion, and tenure, schools of education risk making bad decisions about the composition of their faculties.

• School improvement requires quotidian labor, many persons thinking hard on their feet, responding to an endless flux of concrete situations with knowledge, care, and insight. The myth that fixing the schools somehow depends on the magic of educational researchers deflects attention, material support, and respect from those charged with the real tasks of keeping school, diminishing their morale and confidence.

• Contemporary culture has a great need for thoughtful work that will inform the public consideration of education throughout life and not merely in schools. Relying on educational research for recruitment, promotion, and tenure leads to a great over-production of useless research about schooling, which throttles public communication about education in a din of nonsensical noise. Good schools work poorly in a culture enervated by pedagogical pollutants—excessive inequalities, mendacious leaders, grasping fiduciaries, narcotic entertainments, distorted ideals. Research should lay bare all that dampens educational aspiration, but it avoids this challenge, for it may anger this interest or that, prejudicing its value as a tool in recruitment, promotion, and tenure.

• By working primarily to secure their recruitment, promotion, and tenure, educational researchers become alienated and defensive. They avoid intellectual risks; they become imitative; they seek narrow, predictable specialties; they address a limited readership and nurture small ideas that others will find unexceptionable. In so doing, they educate themselves poorly and as poorly educated educators, they prove impotent as the educators of educators.

Cite This Article as: Teachers College Record, Date Published: March 28, 2007

http://www.tcrecord.org ID Number: 13956, Date Accessed: 4/14/2007 4:48:08 PM

Background/Context: For over two decades, the boundaries between the social sciences and the humanities have become blurred, and numerous articles and books have been written about the infusion of the arts in qualitative research as a means to collect and analyze data and to represent findings. Yet these arts-based research processes, although present in the social sciences, are still largely invisible in a research climate that privileges (e.g., through publication, funding, and recognition) work claiming to be exclusively scientific.

Purpose/Objective/Research Question/Focus of Study: To fully develop the potential of the arts for a transformative educational inquiry, the synthesis of scientific and artistic methods must be fully explicated through clear examples that address theoretical and empirical concerns. This article focuses on explicit arts-based approaches that the authors employed in a 3-year teacher education study of professional conflicts experienced by novice bilingual teachers. Authors describe how they used the arts and to what end, addressing questions of artistic processes, expertise, and research validity.

Research Design: The research design included theatrical and literary techniques alongside more traditional qualitative methods of inquiry (e.g., participant observation, audio- and video-recorded focus group interactions, interviews, and surveys). Authors initiated performative focus groups based on the work of Paulo Freire and Augusto Boal in which participants share and act scenes from power-laden experiences of conflict, rehearsing strategies for personal and social revolution. This embodied data enabled the research team to focus empirical and pedagogical attention on both participants’ physical and verbal “scripts” or trans/scripts: compressed renderings of original transcripts that utilize techniques from poetry and the dramatic arts to highlight the data’s emotional “hot points” and heightened language from the original discourse.

Conclusions/Recommendations: This study illuminated the range of experiences and emotions involved in novice bilingual teachers’ professional lives, signaling the value and validity of research that is both artistic and scientific. Such hybridity may at first appear to make for unexpected and potentially haphazard methodological mergers. The authors do not claim to have resolved these epistemological tensions, but to have exploited both traditional and artistic research methods to broaden the notion of what counts as “research” in teacher education and to conduct research that is engaging to researchers and participants alike.

In the 25 years since Clifford Geertz (1983) declared the boundaries between the social sciences and the humanities blurred,1 numerous articles and books have been written about the infusion of the arts in qualitative research, including recent studies on the use of fiction (Barone, 2001; Kilbourn, 1999), dance (Blumenfeld-Jones, 1995), the visual arts (Irwin & de Cosson, 2004), poetry (Cahnmann, 2003), and other art forms as a means to collect data, analyze data, and represent findings as arts-based educational research (ABER; Barone & Eisner, 1997). A common misconception, however, is that ABER is only useful to those who are trained, professional artists or to educational researchers who have the expertise to produce research-informed short stories, novels, performances, paintings, sculpture, poems, and other pieces of finished, stand-alone art. Although artistic products may provide compelling and creative ways to present research findings to a wide variety of audiences, artistic products as representation are far from the only value in utilizing the arts as a form of inquiry. In this article, we assert that many artistic techniques are inherently part of any scientific process, especially so in qualitative inquiry approaches to data collection, data analysis, and data representation. However, arts-based research processes, although present in the social sciences, are still largely invisible in a research climate that privileges “scientifically based” work (Shavelson & Towne, 2002). To fully develop the creative potential of the arts for a transformative educational inquiry, the synthesis of scientific and artistic methods must be fully explicated through clear examples that address theoretical and empirical concerns. To forefront this discussion, we explore the following questions: What are some of the ways that the arts might be thought of as offering useful tools for all researchers to employ, regardless of artistic training, alongside other research methodologies in the empirical process? What level of competency and/or expertise is required, and what questions should be asked by a researcher interested in adding the tools of the arts to the “toolbox” of methodological possibility?

Because there are still too few examples of how to merge techniques from the arts and social sciences (and explicit justification for doing so), this article shares the ways that we used theatrical and literary techniques alongside more traditional qualitative methods of inquiry (e.g., participant observation, audio- and video-recorded focus group interactions, interviews, and surveys) in a study of bilingual teacher development. We aspired to erase “traditional lines between the realms of social science and the creative arts” (Barone, 2002, p. 255) to foster both a lively professional development experience for study participants and an empirical process that would allow us to “make new, insightful sense of data during and beyond the research project” (Cahnmann-Taylor, 2008, p. 9).

We review a brief history of blurred genres in educational inquiry and share an example of our own teacher education study that has merged qualitative methods with techniques from the literary and performing arts. We identify how this merger informed each phase of our work, including performance-based approaches to data collection, trans/scripting data for analysis, and arts-based decisions in representation.

REVIEW OF BLURRED GENRES IN EDUCATIONAL INQUIRY

In the last few decades, social science researchers and theorists have begun to explicitly recognize the ways in which all social science inquiry that realizes its transformative potential is influenced by both scientific and artistic ways of knowing, including the literary, performing, and visual arts. For example, Heath’s (1983) classic ethnography of children learning to use language in two different communities is a primary example of blurred genre work that implicitly drew on narrative structures and metaphor in a literary approach to ethnography. Clifford and Marcus’s (1986) book Writing Culture collected the first group of essays to explicitly address the poetic and political nature of cultural representation, drawing attention to the literary and rhetorical dimensions of ethnography, a frequently used methodology in educational inquiry.

Music theory and technique have also influenced some of the most noteworthy discourse studies in education: analyzing speech for its rhythm, meter, pitch, and tone. For example, Erickson and Shultz’s (1982) study of counselor and student interactions used musical notation in analysis to discover that distorted rhythms in communication were heavily associated with cultural and racial differences. Erickson, with experience in music composition and theory, used his creativity to enhance his ability to hear and make sense of discordance and harmony in everyday talk. Similarly, Foster’s (1989) study analyzed the musical qualities of an African American teacher’s classroom discourse to shed light on the qualities of her success in an urban community college classroom. In particular, Foster focused on the teacher’s use of church-influenced discourse patterns such as vowel elongation, cadence manipulation, and repetition. Edelsky (1981) addressed the visual and aural aspects of transcription, identifying areas of concern regarding how to best represent the authenticity and dimensionality of an observed interaction for conversational analysis (see Cahnmann & Siegesmund, 2008, for additional examples of blurred genre and arts-based research).

ARTFUL SCIENCE AND SCIENTIFIC ART

The blurred-genre approach represents one way that educational researchers have grappled with what Norman K. Denzin and Yvonna S. Lincoln (1998) refer to as the twin crises of representation and legitimacy in qualitative inquiry. One the one hand, we can no longer assume that the researcher can “directly capture lived experience” (p. 3). On the other hand, we don’t represent truth or reality, but rather create it in our texts. Over the past two decades, postmodern turns in the social sciences call into question the binary between what constitutes “science” as distinct from what constitutes “art.” We draw the reader’s attention to a continuum of inquiry that exists between these two imaginary poles. In the researcher’s imagination, there is what Kuhn (1962/1996) described as “normal science”: that which perpetuates theories and empiricism that are accepted as status quo foundations for scholarly practice. Science, in this status quo sense, is often understood as existing on an opposite pole from “normal art,” akin to binaries of fact-fiction, objectivity-subjectivity, truth-invention, and so on. Like “normal science,” artists too can become bogged down in accepted practices—those that do not demand “the attention of current discovery” and fail to surprise the artist with new ways of seeing and being in the world that are “as the etymology of the word ‘surprise’ literally states, beyond grasp” (Hirshfield, 2007, p. 28).

The division between what is considered art as opposed to science has a long-standing history, one that has limited the possibilities for inquiry. As Richardson (1997) noted,

From the seventeenth century onward, Western science has rejected ‘rhetoric’ (in the name of ‘plain’ transparent signification), fiction (in the name of fact), and subjectivity (in the name of objectivity)’ (Clifford, 1986, p. 5) . . . . Literature was denied truth value because it ‘invented’ reality rather than observing it. (p. 15) Such a binary ostensibly denies scientists’ use of literary devices in representation (Richardson specifically noted the predominance of metaphor in “scientific” research) and artists’ use of observation to collect, analyze, and represent “data.” We posit that science and art in educational research is not the either/or proposition claimed by those issuing formal admonitions against calling arts-based work “research” (Shavelson & Towne, 2002) and denying funding in the name of No Child Left Behind. Clinging to this science/art binary leads many education researchers to avoid mentioning or even using techniques from the arts, dissuaded from doing so either to retain “scientific validity” in the eyes of peer review or to acknowledge feelings of inadequacy in one’s training from an artistic standpoint.

We are interested in the space between what has traditionally been defined as art and science and consequently carry out research practices in education that are informed by tools and techniques from these seemingly opposite fields. We believe that syntheses between the arts and sciences hold the greatest potential for surprise, moving away from what is normal, accepted, and done, and toward what is risky, unexpected, and potentially transformative. For example, where would science be had Einstein not broken with Newtonian conventions of his time? Where would poetry be without great innovators such as Emily Dickinson and Walt Whitman? We can never know these answers because art and science developing within sociohistorical contexts requires ongoing processes of anomaly, novelty, and revolution.

One could say that such a revolution has already occurred in the social sciences, but rarely do we find rich examples of artful science and scientific art explicated in terms of methodology and results. Although aiming to maintain “epistemological humility” (Barone, 2008, p. 11), we share now how we have blurred our methodologies in each phase of our research with novice bilingual teachers. Our work strives to engage in what Barone (2008) calls “conspiratorial conversations” regarding the qualities of teacher education for social justice and critical multicultural reflexivity (p. 39). Through the discussion of our process, we hope to add to the conversation already started by qualitative researchers such as Eisner (1991), Behar (1996), Richardson (1997), Barone (2001), Cahnmann (2003), and others who advocate arts-based methods that expand methodological tools to include those from both the sciences and the arts to make new, insightful sense of data during and beyond the research project.

PERFORMING FOCUS GROUPS FOR EMBODIED DATA COLLECTION

In charge of formatively evaluating a grant program intended to increase the numbers of bilingual, mostly Spanish-English, teachers in Georgia, our professional development and program evaluation team sought meaningful, engaging practices to support novice teachers, especially those for whom English was not a first language and/or for teachers working as advocates for minority populations. We met with bilingual teachers regularly around a table in traditional conversation-based focus groups to talk about (and document) challenges and frustrations en route to becoming fully certified bilingual teachers in the South, where there has been little foundation for bilingual education (Cahnmann, Rymes, & Souto-Manning, 2005). Based on previous work using critical pedagogy (Freire, 1970), critical performance (Boal, 1979), and arts-based approaches to thinking about research methodology (Cahnmann & Siegesmund, 2008), we took our first risks as qualitative researchers: We moved focus group tables and chairs aside to put words into action. Rather than talking about struggles or performing them for teachers, we decided to blur the verbal and performative by getting up from our chairs and reenacting these teachers’ stories collaboratively. Specifically, we initiated performative focus groups based on the work of Paulo Freire (1970) and Freire’s colleague and friend, Brazilian activist/playwright Augusto Boal (1979, 1995), in which participants share and act scenes from power-laden experiences of conflict, rehearsing strategies for personal and social “revolution” (Boal, 1979).

Though we will address this point in more detail later in the article, we must here note the issue of artistic expertise and competency, one of the “tensions” (Eisner, 2008) regarding arts-based inquiry. Though no member of our research team is formally trained as an actor, one researcher had theater experience in college and community groups, and others had attended Theater of the Oppressed (Boal, 1979) workshops with Boal himself or other practitioners. We read books and articles on performance pedagogy, attended several conferences where Boal’s work was illustrated, and rehearsed activities among our research team to achieve competency before changing our traditional focus group forum to one that was more dynamic, embodied, and engaged.

PERFORMATIVE FOCUS GROUPS

Just as we had done in traditional focus groups, we asked participating novice bilingual teachers (total number of participants = 45) to start each session by sharing their struggles and concerns (sessions took place four to six times a year with anywhere from 3 to 23 participants). We asked participants to narrate recurring stories of conflict, positioning themselves as “protagonists” and describing interactions with “antagonist” others. Once stories were shared, the group collectively selected one story that represented a generative theme for the group (e.g., interpersonal conflicts related to race, class, or citizenship status and/or language proficiency). These generative themes of conflict represented diverse situations and included struggles between teachers and parents, colleagues, administrators, students, and college professors (see Figure 1). Generative themes that proved most relevant and/or most urgent to the collective group of novice bilingual teachers were codified and performed.

The specific details of the generative case (e.g.. a racist or unethical colleague, a belligerent parent, an epistemological bully for a professor) are used to create a two- to three-scene drama, but the selected story no longer belongs to the initiating storyteller but to the collective, whom Boal refers to as “spect-actors.” Boal (1979) coined the term spect-actor to break the divide between spectator and actor. Although the initial storyteller might first play the role of the protagonist, he or she is frequently replaced by other spect-actors in the group who identify with the protagonist and who perform alternative courses of action to change the outcome of the conflict. Thus, rather than simply giving verbal advice to the storyteller, as typically occurs in the problem-solution “banking model” (Freire, 1970) format of most teacher education contexts (e.g., “Why don’t you try talking to your principal about this?”), spect-actors are encouraged to share the story by living it themselves, getting out of their seats, stepping into the protagonist’s role, and performing (e.g., becoming the teacher who schedules an appointment with the principal and role-playing themselves in the imagined dialogue). After each enactment, the facilitator or “difficult-ator” of the session, whom Boal (1979) refers to as “The Joker,” asks for a “reality check” to collectively determine the viability of the proposed strategy.

Believing that “where there is power, there is resistance” (Foucault, 1978, p. 95), we engaged spect-actors in rehearsals of these collective narratives to perform power as a “multiplicity of force relations” that are moving, unstable, and omnipresent (pp. 92–93). Spect-actors often accepted or rejected strategies based on whether they “felt right” (e.g., “I couldn’t say/do that, I’d lose my job!”), which would then lead to further enactment, strategizing, and reality checks in a recursive model of this performance-based focus group format (see Figure 2).

This rearticulation of power makes it so that one is not powerful or powerless, but rather always involved in relations of power that are discursive and productive. Lib Spry (1994) described shifting the language of “oppression” in her theater work with largely White middle-class Canadians to the language of “liberation,” preferring to ask participants who has power-over them rather than who oppresses them.

By introducing this term “power-over” (Starhawk, 1987, p. 9) into my TO workshops rather than using the vocabulary of oppression, I have found that people are more prepared to look at the power structures in which they live and the role they play in it. Understanding these structures is a first step towards change. (pp. 174–175)

MARISOL’S CASE

One case example from this type of performative focus group illustrates how participants performed problems and options for resolution. Marisol (all names are pseudonyms), an English for speakers of other languages (ESOL) teacher of Cuban origin, performed a recurring problem that she had with a colleague who often shopped online during instructional time and appeared not to care about the ESOL students in the colleague’s mainstream second-grade class. Her oppressive situation resonated with other bilingual teachers’ experiences with colleagues who were perceived as similarly insensitive, disrespectful, or racist toward English language learners. For this reason Marisol’s story was chosen for its generative theme, or issue for collective engagement: how to be the most effective advocates for ESOL students amid colleagues whose attitudes and actions were perceived to undermine advocacy/best practice. The codification included breaking the story into scenes, one scene between Marisol and her students, and two scenes between Marisol and her colleague, in which the result was no change in the protagonist-teacher’s sense of dejection.

Marisol was the first to perform as “The protagonist ESOL teacher,” and then described feeling frustrated and angry and wanting “to choke” her dismissive colleague [Line 5], aware that this was not a “real” option for resolving conflict. A peer bilingual teacher in the group, Jorge, wanted to suggest what Marisol might do to change her colleague’s attitudes and/or behavior [Line 6]. In this performance-based focus group, telling someone “what to do” was insufficient. The arts-based researcher immediately directed Jorge to spect-act “as if” he were the ESOL teacher, performing his problem-posing strategy [Line 7]:

Scene: Show Don’t Tell

1. Antagonist Teacher ((looking away from her computer and speaking to Marisol, who returned the ESOL students to class)): Yes. Keep the ESOL kids, keep them all! ((Antagonist spreads her hands away from her to emphasize all of the children))

2. Marisol: Thank you. I will! ((hiding frustration))

3. Antagonist: Okay, great!

4. Arts-Based Researcher: Stop! ((to Marisol)) What are you feeling when she [the antagonist] says that?

5. Marisol: That I wanna choke her. ((laughter from group)) Because it’s not like she’s, like, “Oh yeah, someone is doing something good for them!” It’s “Thank God, I don’t have to deal with them, I don’t even have to look at ‘em.”

6. Researcher: Does anyone have an idea, a strategy for change? Okay, ((to Jorge)) you do? Replace Marisol. ((Jorge hesitates, raises his hand to speak his idea))

7. Researcher: You, you need to do it.

8. Jorge ((almost inaudibly in background)): Okay. ((Jorge rises to take over Marisol’s “role” as the ESOL teacher))

9. ((Beginning with Scene One. Antagonist has resumed her role and is sitting in a chair in the center of the room. Jorge is standing a few feet away. Scene begins.))

10. Jorge: ((To the antagonist)) Hello, how are you?

Jorge was one of many participating teachers who performed various strategies for communicating through this conflict, including everything from the practical (e.g., documenting the antagonist’s behavior) to the ridiculous (e.g., destroying the antagonist teacher’s computer to prevent her from shopping online during instructional time and ignoring the ESOL students). Despite solutions that didn’t meet the reality check criteria for action, the more ridiculous strategies seemed to strengthen group identity and encourage more creative, out-of-the box performances.

EMBODIED DATA

In these focus groups, we found that the fusion of arts-based and qualitative approaches to data collection (theatrical techniques and the focus group as a method, respectively) were valuable in a variety of ways. First, we recognized stages of the performative focus groups as a form of recursive qualitative research. Sharing of teacher struggles through codification (e.g., dividing up a “story” into two to three performative scenes), spect-acting (rehearsing the same scenes with different actors playing different potential interventions), and reality checks (ascertaining the feasibility of proposed interventions in “real” life) were part of a recursive process of data collection, analysis, representation, and collective credibility-testing. This arts-based conception of our focus groups allowed us to engage in an embodied and reciprocal empirical process that was meaningful and engaging to participants and researchers alike.

Additionally, the focus group data captured much richer, more nuanced detail because of the collective dynamic established through performance, blurring verbal narrative and performance genres. Although Marisol’s story could have been easily explained and understood in words, a wholly verbal explanation would have lacked the visceral punch of her reenactment with the antagonist. Fellow spect-actors related to Marisol, perhaps not for the specifics of her case but because of the visible emotions of frustration that performing the life of the “idealistic new teacher” amid cynical colleagues brought forth. This sympathetic turn elicited other spect-actors’ personal narratives and performances.2 In several circumstances, a spect-actor spontaneously offered a verbal suggestion for the protagonist, but when asked to embody the strategy in performance, the spect-actor declared that his or her suggestion didn’t work because it didn’t “feel right.” This emphasis on feeling and the body became not only the site of enactment and credibility but also a suggestion for change. In Marisol’s case, one spect-actor stood uncomfortably close behind the seated mainstream teacher and peered over her shoulder at the computer screen. In another conflict, spect-actors proposed to a diminutive bilingual teacher struggling with a larger, threatening colleague that she stand up straight and raise her hands upward, running fingers through her hair in an attempt to make herself feel and look taller.

Scene: Be a Lion!

Arts-Based Researcher #1: I’ve noticed every time you perform talking to your colleague you’ve been doing this ((hunching shoulders forward and downward)).

Sonia [Protagonist Teacher]: Yeah, I know! I’ve been intimidated from the first day.

Doris: Why don’t you try putting your hand to, your hands up—make yourself higher. Your hands can help you go higher than she is.

Sonia: Oh, really? I never heard of that—this is good, let’s see if I can try that.

Arts-Based Researcher #2: That’s what we say with mountain lions, too—if you see one on your path, don’t run or crouch but rise up and make yourself bigger. Be the lion! Let’s try it again.

Sonia: ((having second thoughts at acting it out on-the-spot)) I don’t think I can do it.

Doris: You don’t even have to put your hand up there, just feel like there’s hair in your face and you move your hands to brush it away.

Sonia: ((motions her hands towards her face)) Oh, okay.

((Group laughter)).

Regardless of whether suggested strategies were deemed “real,” we’ve found that the process of reenactment itself promotes solidarity-building and is preparatory and agentive. Throughout transcripts, we see frequent bursts of laughter (such as the one in the previous case) expressed during group rehearsals. We have found that laughter serves as a form of dynamization, arousing the group to “create new actions, new alternatives which are not substitutes for real action, but rehearsals, pre-actions . . . [for] a reality we are trying to change” (Boal, 1995, p. 72). Boal distinguished this kind of cathartic release of tension from what he referred to as “Aristotelian catharsis,” which disempowers, tranquilizes, and purifies the desire for change. In contrast, the group-generated laughter represented cathartic releases of tension through performance, having the potential to release spect-actors from detrimental blocks to change both in the performance and in real life.

Thus, embodied data enabled our research team to focus empirical and pedagogical attention to both participants’ physical and verbal “scripts” or “trans/scripts” (discussed next), adding to the value and credibility of our work. During reality check moments, participants decided whether alternative performances had value in their own specific contexts. Credibility was ascertained through multiple iterations of performance-based workshops in which the universality and particularity of specific teacher conflicts were ascertained.

Being open to data collection procedures that were both artistic and scientific allowed us to engage new teachers in dynamic, performative conversations that were meaningful to both our research interests and participating teachers’ ongoing concerns. We were struck by the utility of performance-based focus groups. Participants in anonymous evaluations commented on their engagement and pleasure in this new focus group form. In Likert scale surveys, 68% of attendees classified the focus groups as very useful for professional development, and 32% deemed them useful, with no participant rating the experience below a 4 on a 1–5 scale. Many praised the performance work in contrast to seat- and text-based work more customary in teacher education environments. Following is a representative example from anonymous written evaluations: “Problems and situations [enacted] were directly relevant to real classroom happenings and they were very useful to cope with real problems if they arise later. This is something that the classes don’t teach in the coursework, but every teacher faces these conflicts [italics added].”

We found an arts-based approach to data collection to be immediately useful to participants who collectively rehearsed embodied options for overcoming struggle rather than simply hearing “solutions” from us as researchers and teacher educators. The act of physically rehearsing different strategies allows spect-actors to demechanize the body’s ways of being, exploring ways to act “as if” one had the power to change situations perceived as relentlessly disempowering. This performance-based approach to teacher education research lends itself to critical problem-posing as opposed to problem-solving because acting out the variety of alternatives to context-specific issues allows teachers, like skillful actors, to improvise rather than be stuck in one proscribed “role” or following one proscribed “script.” Teacher-generated performances seemed complementary, and, at times, more pertinent than multicultural foundations texts (e.g., Nieto, 2005) more commonly found in professional development courses because performances grounded multicultural theory in the texts of participants’ lives.

ANALYSIS AND REPRESENTATION OF TRANS/SCRIPT DATA

Initially, we took a discourse-analytic approach to transcripts of our seated focus group sessions, combined with analysis of qualitative interviews and evaluation surveys (Cahnmann et al., 2005). We used qualitative software programs (e.g., Transana and atlas.TI) to transcribe and analyze verbal and nonverbal interactions, identify emergent themes, and capture still images and video clips to share with participants during follow-up interviews. Initial discourse analysis of more traditional, seated focus groups underscored the need to change the way in which we were eliciting information from participants because these first focus groups failed to challenge beliefs in traditional texts and institutional roles as legitimizing knowledge (e.g., dictionaries and test scores to define who is or is not “Latino,” “bilingual,” or “qualified to teach”). Further, through the discursive analysis of initial focus group data, we were led to investigate teachers’ experiences as texts, including a framework that would be dependent on both the verbal and nonverbal performance and that would present interactional components.

Our hybrid methodology continued our use of qualitative analytic tools and included methods that were arts based. We began inductively coding transcriptions and interview data and looking for themes in the variety and quality of strategies that teachers performed to resolve conflict. Our coding analysis helped document patterns of intervention but failed to yield the emotional feeling of the focus groups and was not useful for us in sharing data back with our group of bilingual teachers.

To capture the emotional qualities in the project and to understand our data more fully, we also responded “artfully,” crafting what we refer to as trans/scripts: compressed renderings of original transcripts that utilize techniques from poetry and the dramatic arts to highlight emotional “hot points” and heightened language from the original discourse in our data. Although this combination of approaches may at first appear epistemologically confused, we believe that these choices enabled complementary understandings. Artful trans/scripts allowed us to revisit the data, capturing the lived feeling of the collective performance. Coding the performance data helped us generalize across multiple cases and understand patterns. Throughout this article, we artfully render trans/scripts in ways that we feel are faithful to what was originally spoken but that use literary license (e.g., removing excessive or redundant spoken language) to convey (we hope) the emotional qualities of the original experience.

Next is an example trans/script that details the language and body movement of Marisol’s initial “drama” of struggle when working with a colleague who appeared to resent newcomer ESOL populations in her classrooms.

Trans/script: Marisol’s Case, “With colleagues like this, who needs enemies?”

Scene One: What are you doing in class?

((Marisol Jiménez is with her second-grade ESOL students, ready to take them back to their homeroom teacher. She engages Angel in discussion.))

Ms. Jiménez: ((To the class)) Children, let’s get ready to go back to class. Tell me, Angel, what are you learning in Mrs. Smith’s classroom?

Angel: Nada.

Ms. Jiménez: What do you mean nada? Nothing? You must be learning something.

Angel: Ms. Jiménez, we don’t learn in that class. Mrs. Smith just sits there all day at her computer ((mimics typing on a keyboard)).

Scene Two: Not even hello.

((Ms. Jiménez takes her students back to class. She stands at the door to Mrs. Smith’s classroom and waits to engage her. Mrs. Smith doesn’t look up from her computer. Ms. Jiménez feels dejected and leaves the room without engaging her colleague.))

Scene Three: Take them! Take them ALL!

((Ms. Jiménez returns the next day to Mrs. Smith’s classroom and tries to get her attention. After several seconds, Mrs. Smith finally looks up from her monitor.))

Ms. Jiménez: ((To Mrs. Smith)) Good morning.

Mrs. Smith: Oh, is it a good morning? A good morning is the crossword and a refillable cup of coffee! ((laughs and turns back to her screen))

Ms. Jiménez: Mrs. Smith, my students have made a lot of progress in reading and I’d like to celebrate this week with a pizza party. I was wondering if it’d be alright to keep them through lunch this Friday?

Mrs. Smith: Keep them? You want them on Friday, keep them, keep them ALL! All day if you want, no problem with me.

Ms. Jiménez: Thank you. ((leaves the room dejected, angry, and feeling isolated))

In discourse or conversation analysis, transcripts are often marked by precise symbols to approximate as closely as possible the original stress patterns, intonation, overlapped speech, and pause length upon which analysis is drawn. In contrast, the above three trans/script-based scenes are a compression of several pages of original transcript, and for purposes of clarity and recursive utility in teacher education, we sacrificed discourse analytic symbols for dramatic ones (e.g., stage directions, compressed language, and so on). Our arts-based approach to analysis illuminates dramatic elements of conflict that resurface in bilingual teachers’ lives.

Because manipulation of actual transcript data often occurs implicitly in much qualitative and discursive analysis, here we deconstruct the term trans/script to explicitly acknowledge working with discourse as a “rough draft,” producing compressed ethnodramatic renderings (Saldaña, 2005) that echo what Alfred Hitchcock is known to have said of making movies—to present real life with the boring parts taken out. Translation, transformation, transference, and so on—trans is a Latin prefix meaning to go across, through, on the other side, and beyond. All trans/scription goes beyond what was originally said to render spoken and embodied communication in written form. Arts-based approaches to analysis encourage the researcher to make the trans part of trans/scription apparent, utilizing tools from both the arts and sciences to render the original impact and feeling on the page and to guide the researcher to focus points in the text.

Through analysis and trans/scription of our improvisations, we were able to rescript open-ended scenes. Trans/scripts become both product and process, used and rescripted with groups of teachers in different contexts for addressing tensions and strategies that critical, multicultural educators employ in the midst of conflict and social change. In addition to trans/scripts, we have used open-ended vignettes based on generative themes—encouraging participating teachers to trans/script their own scenes based on the real-life connections. Next is one such vignette that has appealed to groups of educators in a wide range of grade levels and subject areas:

Open-Ended Vignette: “‘You are nobody’: Working with a student that doesn’t respect your authority”

I am disheartened after working with a student that continuously misbehaves and disrupts my elementary school class. Despite several administrative interventions, the student still is giving me problems. When I try to reprimand him, he says “You are nobody. My mom can come in here and say anything she wants.” I don’t feel like I have the skills to handle him, and I am exhausted by the power struggles in my classroom.

A group of 23 teachers worked in smaller groups of 3 or 4 spect-actors and created unique versions of this commonly experienced tension regarding student participation and behavior. Each group’s performance became a new transcript for generating new possibilities for action. As researchers, our responsibility is to produce trans/scripts from transcripts (e.g., refined renderings of recorded interactions as vignettes or scripts) so that they can be used in a recursive process to promote new performances, new transcripts, and new trans/scripts for future use. The following trans/script was created from a transcript of Roberta’s performance.

Trans/script #1: Ms. Roberta Salas’s case: “You are nobody”

Scene One: What do you mean you didn’t do it?

((Roberta Salas is with her fourth-grade students, checking homework assignments.))

Ms. Salas: [To the class] Children, take out your homework. ((In front of the first student’s desk)) Very good, Iliana, you’re making progress. ((Continues praising others)) Very good. Thank you, very good. Yes, excellent, María. ((Stops in front of Dana’s desk)). Dana, where’s your work?

Dana: ((shrugs her shoulders)) I didn’t do it.

Ms. Salas: What do you mean you didn’t do it? Why not?

Dana: I didn’t feel like it, that’s why. ((turns toward her neighbor to talk))

Scene Two: To the principal’s office!

Ms. Salas: Dana, we’ve been going through this all year long. It’s not acceptable for you not to do your homework and not to respect this classroom. We’re going to the principal right now.

Dana:Fine, what do I care?

Classmates: oooooohhhhhhh!

Scene Three: Not the Desired Outcome.

Ms. Salas: Dana, I just spoke with the principal, and we called your mother. We’ve decided to give you a suspension for three days.

Dana: YES!!! ((pumps her arm with enthusiasm))

As a group, we rehearsed these three scenes, encouraging spect-actors to intervene and change an aspect of the protagonist teacher’s behavior that might also influence Dana’s response and participation. Documenting in-flight decision-making and the quality and variety of interventions helped the research team analyze strategies that teachers can draw on during moments of struggle, helping us to see the common and the unusual. Since initial coding of teacher strategies, we have found interventions to include “calling the boss” (e.g., calling on an administrator for support); creating networks with other colleagues; keeping careful documentation (e.g., dated journal notes, evidence); avoiding confrontation and enduring the struggle to its inevitable end (e.g., a different semester, a change in workplace); and open confrontation with an antagonist.

In addition to rehearsing options for behavior during moments of conflict, the group also rehearsed what Boal (1995) referred to as the “Rainbow of Desire”—that is, the many different hues that one individual contains in any challenging interactional moment. In this case, spect-actors modeled the many different emotions and feelings a teacher has during a conflict with a student. Spect-actors were encouraged one by one to join an ensemble of human sculptures portraying the many hues that Ms. Roberta Salas has when interacting with students like Dana. Thus, our analysis included the quality and variety of portraits that teachers created to understand their antagonists and how these nuanced portraits influenced the types of interventions perceived as available to them.

Trans/script: Rainbow of Desire

Arts-Based Researcher: We’re going to do what Boal calls Rainbow of Desire. The idea is that we need to understand the complexity of our characters. There are multiple behaviors, emotions, and histories that inform the present moment. Please come up one volunteer at a time. What is that teacher, Ms. Salas, feeling? Sculpt a silent image that would express it.

((Rona comes onto the stage and performs strangling Dana.))

Arts-Based Researcher: Freeze. That’s one. ((Laughter from some in the group)) Someone else?

Roberta: Yeah. ((Poses with her hands in the air like she’s had enough)) I’m frustrated ((Points to herself)), completely frustrated.

Arts-Based Researcher: Good, another!

Lisette: ((Sits in the seat next to Dana, leans over so that her right ear is near Dana’s chest)) I’m trying to listen to what’s in her heart.

Many Voices: Yes! Yes!

Linda: ((Lies down on the floor at the edge of the audience)) So can I stay over here ‘cause I’m home?

Arts-Based Researcher: Sure! ((Nodding to Linda))

Linda: I’m sleeping and I’m dreaming. I’m waking up, and I’m trying to figure out how to solve the problem in my sleep. ((As she speaks, she rises to a sitting position with left hand to chin—in a pondering gesture))

Angela: That happens to me too!

Many voices: ((General agreement))

Rona: I want to say something. I’m not going to choke the child if they do this to me. And they do do this to me sometimes. I mean what you do and what you feel are two different things.

For a long time, critical pedagogy has relied on language to create possibilities and has fallen short of embodied social action (Pineau, 2002). As Boal designed it, Theater of the Oppressed (including Forum Theater, Rainbow of Desire, and other T.O. techniques) opens up a space for critical performative pedagogy and embraces a language of possibility as participants engage performatively in contexts in which the qualities of the “protagonist” and “antagonist” are multifaceted and contextually situated. Just as Rona stated, what we do and what we feel are often very different. Although there are several approaches to teacher education that espouse “reflective teaching,” these are often very individual processes whereby a novice teacher reflects in isolation through a journal or in paired collaborations with a mentor teacher or university supervisor. Seldom are arts-based opportunities available for new teachers, especially those who are working on behalf of marginalized communities, to reenact their experiences in a collective, rehearsing and validating the emotional hues that inform embodied practice. Rarely are there opportunities in teacher education contexts for teachers to express the range of emotions they experience during the school day in a way that feels safe, honest, and validated. Even rarer are opportunities for researchers to engage artistically with teachers to document the range of experiences and emotions involved in their professional lives.

ARTS-BASED RESEARCH IN EDUCATION: REFLECTIONS ON POSSIBILITIES AND CAUTIONS

If arts-based research culminates in little more than a delightful poetic passage or a vivid narrative that does little educational work, it is not serving its function. In other words, I am trying to remind us here that in the end research is an instrument, whether arts-based or not, that is supposed to contribute to the quality of education students receive and that arts-based research must ultimately be appraised on the extent to which that aim is realized. (Eisner, 2008, p. 23)

Our project has merged practices and theories from postmodernism, critical pedagogy, and critical performance; in so doing, we believe that arts-based performance research does respond to Eisner’s stated concern and contributes to the quality of teacher education and, ultimately, to the quality of education that students receive. Through interactive, co-constructed design, we have demonstrated how using techniques from the literary and performing arts has helped us create meaningful, engaging contexts for collecting and analyzing research with novice bilingual teachers. The process of arriving at this syncretism, however, was not trouble free. In conclusion, we address some of the struggles and concerns that have arisen during our arts-based research process, notable among them our own experience in the arts and the recursive nature of this work.

EXPERIENCE

We began taking risks and using the dramatic and performing arts in our research process with varying degrees of skill and experience as actors, directors, playwrights, and educational researchers. Although all members of our team hold graduate degrees in language and education, none of us has an MFA degree in theater arts, and none of us claims to be producing Broadway-, Off Broadway-, or even Off-Off-(Get-Off!) Broadway-style productions or hope to take our trans/scripts to the literal stage. However, the more we experiment with theatrical techniques, the more we build our competence in both the arts and social science methodologies, working toward expertise. Researchers from this project have attended the annual Pedagogy and Theater of the Oppressed workshops with Boal offered in different locations throughout the country. We have invited guest speakers to build our knowledge of Boalian theater games and craft, and we hold our own “rehearsals” to develop our dramatic as well as our data-gathering and analysis skills. Much like practicing poets, playwrights, and fiction writers, we also participate in writing workshops to reflect on how we represent our empirical work and with what effect. Such contact and practice is necessary, for, as Geertz (1983) reminded us, researchers drawing on the arts are “drawn into the rather tangled coils of its aesthetic” (p. 27). Although we believe that the arts are a method open to all researchers, we also believe that the arts, like scientific methods, require time, commitment, and the pursuit of training.

RECURSIVITY AND NOT KNOWING WHAT COMES NEXT

Another challenge, or opportunity, that we encountered was a reappraisal of the research process. Rather than the linear mode of inquiry so often detailed in qualitative handbooks and studies, our work with performative focus groups and trans/scripts was dizzily recursive. Initially, we approached this process in a linear fashion. However, when we began trans/scripting our transcripts for analysis, additional data collection, and representation with our group of bilingual spect-actors, we began to question the discrete categorizations of collection, analysis, and representation. As mentioned previously, trans/scripting is both process and product. As we look at the video-recorded data and written transcript, we make authorial decisions to create the trans/script or vignette that highlights key themes or “hot points” in the data. Deciding where the hot points are and how to fan them for maximum evocation is the process of analysis. The completed trans/script, however, is not merely an artistic representation of data analysis or findings; rather, it becomes data to be analyzed by spect-acting participants in later focus groups. Specifically, group members analyze trans/scripts and/or vignettes by relating them to their own experiences and selecting details of those incidents to dramatize and then represent their analysis to the group. Then the group checks the representational reality, and additional scripts are enacted. In other words, no part of the performative process is static or discrete. As researchers, we are always uncertain as to where the T.O. performative model (Figure 2) and the areas of qualitative inquiry will intersect and/or loop back.

The uncertainty of not knowing what will come next and being unable to easily categorize the research process is a challenge, yet working with the unpredictable is invigorating both as researchers and teacher-educators. As teacher-educators, we did not have absolute answers to teachers’ real, day-to-day concerns: for example, how to work with an uncooperative, unstable, or unsafe colleague, and how to manage students who are disengaged, disrespectful, or even violent in the classroom. Through performance-based focus groups, we learned that not having the answers can be a strength and allow participants’ and researchers’ collective wisdom to come forth and unforeseen challenges to be known.

CONCLUSION

Attributed to legislation such as No Child Left Behind, as well as county- and school-mandated testing and curriculum, teachers and teacher-educators feel constricted in both the content and form that education research takes. Arts-based approaches to research in teacher education are a form of experiment that pushes beyond the limits of what is known, acceptable, feasible, and done, asking participants to explore options rather than solutions and to rehearse the possible rather than the perceived inevitable.

Though the subjects and actors may change, situations of struggle recur in teachers’ lives. New approaches to critical teacher education research are needed that raise awareness to the moving and omnipresent “multiplicity of force relations” (Foucault, 1976/1978, pp. 92–93) that operate in teachers’ lives. Analysis of project trans/scripts provides evidence of the dynamic potential for blurring boundaries between the arts and social sciences in qualitative teacher education research.

Epistemological tensions between artistic and scientific ways of knowing the world may at first appear to make for unexpected and potentially haphazard methodological mergers. We do not claim to have resolved these tensions, but to have exploited both traditional and artistic research methods to understand what it means to diversify our teaching force, including bilingual and bicultural teachers and students in K–12 public schools.

As more researchers become skillful in merging artistic and scientific approaches to empiricism and explicitly share their processes with larger audiences, we hope that current epistemological tensions will give way to new and unexpected mergers that shed light on recurring educational concerns in the pursuit of excellence and equity. We hope that our use of blurred methodologies for data collection, analysis, and representation have documented and communicated this work in ways that reveal its potential and possibility.

Through increased documentation and articulation of these hybrid methodologies, we may see more tables and chairs pushed to the periphery of the teacher education focus group conversation, with more opportunities for engaged, embodied, and critical learning.

Acknowledgements

The authors would like to thank the U.S. Department of Education for support of the Teachers for English Language Learners (TELL) grant through the Transition to Teaching program. We also thank Dr. Betsy Rymes, Esperanza Mejia, Betsy Wang, Meghan Kicklighter, Constanza Beron, Cori Jakubiak, Carmen Cerda, and Paula Mellom for their support. Much gratitude to Terry Osborn for reviewing an early version of this manuscript.

Notes

1 In Local Knowledge: Further Essays in Interpretive Anthropology (1983), Geertz declared that a certain melding between the social sciences and the humanities has been omnipresent in intellectual history. According to Geertz (1983), texts cannot be easily sorted into factual or literary categories after the interpretive turn because they do not fit solely into one category or another. However, the sheer volume of 20th-century texts that would become uneasily categorized as factual or literary suggests that “something is happening to the way we think about what we think about” (p. 20). This interpretive turn, in which discursive resources so readily available to those in the humanities are used explicitly and with new purpose in the service of the social sciences, becomes visible as “society is less and less represented as an elaborate machine or a quasi-organism and more as a serious game, a sidewalk drama, or a behavioral text” (p. 23).

2 Boal (1995) is adamant that replacing the protagonist while spect-acting is not predicated on empathy or being penetrated by the emotions of another, but rather on sympathy—that is, projecting one’s own emotions on the scene and placing one’s self in a situation that resonates with the spect-actor. Spect-actors are not “walking in the shoes” of the protagonist, but rather walking the same road as the protagonist.

References

Barone, T. (2001). Touching eternity: The enduring outcomes of teaching. New York: Teachers College Press.

Barone, T. (2002). From genre blurring to audience blending: Reflections on the field emanating from an ethnodrama. Anthropology and Education Quarterly 33, 255–267.

Barone, T. (2008). How arts-based research can change minds. In M. Cahnmann & R. Siegesmund (Eds.), Arts-based inquiry in diverse learning communities: Foundations for practice (pp. 29–50). London: Routledge.

Barone, T., & Eisner, E. W. (1997). Arts-based educational research. In R. M. Jaeger & T. Barone (Eds.), Complementary methods for research in education (pp. 73–94). Washington, DC: American Educational Research Association.

Behar, R. (1996). Vulnerable observer: Anthropology that breaks your heart. Boston: Beacon Press.

Blumenfeld-Jones, D. S. (1995). Dance as a mode of research representation. Qualitative Inquiry 1, 391–401.

Boal, A. (1979). Theater of the oppressed. London: Pluto Press.

Boal, A. (1995). Rainbow of desire: The Boal method of theatre and therapy politics (A. Jackson, Trans.). London: Routledge.

Cahnmann, M. (2003). The craft, practice, and possibility of poetry in educational research. Educational Researcher, 32(3), 29–36.

Cahnmann, M., Rymes, B., & Souto-Manning, M. (2005). Using critical discourse analysis to understand and facilitate identification processes of bilingual adults becoming teachers. Critical Inquiry in Language Studies: An International Journal, 2, 195–213.

Cahnmann, M., & Siegesmund, R. (Eds.). (2008). Arts-based inquiry in diverse learning communities: Foundations for practice. London: Routledge.

Cahnmann-Taylor, M. (2008). Histories and new directions. In M. Cahnmann & R. Siegesmund (Eds.), Arts-based inquiry in diverse learning communities: Foundations for practice (pp. 3–15). London: Routledge.

Clifford, J., & Marcus, G. E. (1986). Writing culture: The poetics and politics of ethnography. Berkeley: University of California Press.

Denzin N., & Lincoln Y. (1998). Introduction. In N. K. Denzin & Y. S. Lincoln (Eds.), The landscape of qualitative research: Theories and issues (pp. 1–34). Thousand Oaks, CA: Sage.

Edelsky, C. (1981). Who’s got the floor? Language in Society, 10, 383–421.

Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. New York: Macmillan.

Eisner, E. W. (2008). Persistent tensions in arts-based research. In M. Cahnmann & R. Siegesmund (Eds.), Arts-based inquiry in diverse learning communities: Foundations for practice (pp. 16–27). London: Routledge.

Erickson, F., & Schultz, J. (1982). Counselor as gatekeeper: Social interaction in interviews. New York: Academic Press.

Foster, M. (1989). “It’s cookin now”: An ethnographic study of the teaching style of a successful Black teacher in an urban community college. Unpublished doctoral dissertation, Harvard University, Cambridge, MA.

Foucault, M. (1978). The history of sexuality, Vol. 1: Introduction (R. Hurley, Trans.). New York: Vintage Books. (Original work published 1976)

Freire, P. (1970). Pedagogy of the oppressed. New York: Continuum.

Geertz, C. (1983). Local knowledge: Further essays in interpretive anthropology. New York: Basic Books.

Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. New York: Cambridge University Press.

Hirshfield, J. (2007). Poetry and the constellation of surprise. Writer’s Chronicle, 40(2), 28–35.

Irwin, R., & de Cosson, A. (2004). A/r/tography: Rendering self through arts-based living inquiry. Vancouver, British Columbia, Canada: Pacific Educational Press.

Kilbourn, B. (1999). Fictional theses. Educational Researcher, 28(9), 27–32.

Kuhn, T. (1996). The structure of scientific revolutions. Chicago: University of Chicago Press. (Original work published 1962)

Nieto, S. (2005). Affirming diversity. Mahwah, NJ: Erlbaum.

Pineau, E. L. (2002). Critical performative pedagogy. In N. Stucky & C. Wimmer (Eds.), Teaching performance studies (pp. 41–54). Carbondale: Southern Illinois Press.

Richardson, L. (1997). Fields of play: Constructing an academic life. New Brunswick, NJ: Rutgers University Press.

Saldaña, J. (2005). Ethnodrama: An anthology of reality theatre (pp. 166–178). Walnut Creek, CA: Altamira Press.

Shavelson, R. J., & Towne, L. (2002). Scientific research in education. Washington, DC: National Academy Press.

Spry, L. (1994). Structures of power: Toward a theatre of liberation. In M. Schutzman & J. Cohen-Cruz (Eds.), Playing Boal: Theatre, therapy, activism (pp. 171–184). London: Routledge.

Cite This Article as: Teachers College Record Volume 111 Number 11, 2009, p. 2535-2559

http://www.tcrecord.org ID Number: 15439, Date Accessed: 7/29/2011 12:53:55 PM

 Social Reproduction of Inequality: The Racial Composition of Feeder Schools to the University of California

 by Robert Teranishi & Tara L. Parker - 2010

Background/Context: Previous research has pointed to the dismal state of secondary education in California and its implications for college access. These studies identified the extent to which disparities in access and outcomes in California are correlated with school demographics and the relative levels of segregation that exist in California's public high schools.

Purpose/Objective/Research Question/Focus of Study: Our research advances understandings of this association within California's most selective sector of public higher education: the University of California. Three research questions guided our study: (1) In what ways, if any, are the racial compositions of high schools different for first-time freshmen (FTF) at different UC campuses? (2) How are the racial compositions of high schools that are the source of underrepresented minorities (URMs) at each UC campus different, if at all, from the racial compositions of high schools that are the source of White FTF? (3) Do the racial compositions of public high schools in California have an independent net effect on the likelihood that a school will send graduates to the University of California and individual UC campuses, controlling for other school characteristics?

Research Design: To answer our research questions, this quantitative study used descriptive correlations and multiple regressions to examine the high school source of FTF for 8 out of 9 UC campuses for the 2000-2001 academic year.

Findings/Results: Although the University of California as a whole may appear to be making gains in providing opportunities for students of color, accessibility to individual UC campuses for Black and Latino students varied depending on URM or White high school attendance. At 5 of the 8 campuses in this study, a greater proportion of URM FTF originated from White schools, as opposed to URM schools, where they were most concentrated.

Conclusions and Recommendations: Educational policies and practices in the UC system that hold some campuses accountable for goals of equity yet allow other campuses to exclude whole groups of students based primarily on the high school they attended are neither effective nor equitable. Researchers, policy makers, and educational leaders should examine the ways in which admissions policies may inherently privilege one group (those from White schools) and disadvantage other groups (those from URM schools). This problem needs to be addressed by both policy makers and practitioners in higher education and the K-12 system.

As we mark the 55th anniversary of the landmark Brown v. Board of Education decision, the ability for schools to combat racial segregation and send students at equal rates to higher education continues to be challenged. Most recently, the United States Supreme Court ruled on two school desegregation cases1 and limited the ability of school administrators to desegregate our nation's public schools. These decisions, as well as the 2001 ruling of the U.S. Supreme Court2 on the ways in which colleges and universities can use affirmative action, challenge the ability of higher education to account for racial diversity as a compelling interest to our society (Hurtado, Milem, Clayton-Pedersen, & Allen, 1999). California, Washington, Florida, Georgia, and, most recently, Michigan face an added challenge in diversifying their most selective institutions following the end of any use of race as a factor in their admissions processes (Hunt, 2005).

These significant changes to long-standing social policy related to access and mobility in higher education are occurring while many public universities are raising admissions standards for high school graduates, becoming exponentially more selective, and serving an increasingly narrower sector of our society. Perhaps one of the most poignant illustrations of how the combination of these issues is affecting access to higher education can be found in the state of California. The Chief Justice Earl Warren Institute on Race, Ethnicity and Diversity at the University of California, Berkeley, recently released a compelling policy report, "California at the Crossroads: Confronting the Looming Threat to Achievement, Access, and Equity at the University of California and Beyond." Among the key findings are that the University of California (UC) disproportionately serves the state's highest income and best educated families while limiting access for low-income students of color. In addition, the study indicates that half of UC's freshmen come from high schools that educate only one fifth of California's public secondary school graduates.

Other research confirms the dismal state of access to the UC (Allen, Bonous-Hammarth, & Teranishi, 2002; Chang, 2000; Martin, Karabel, & Jaquez, 2005; Teranishi, Allen, & Solorzano, 2004). These studies point to the condition of secondary education in California and its implications for postsecondary preparation, eligibility, and college-going rates, which vary widely for students of different racial backgrounds. Chang found that unequal resources and opportunities in California's high schools influenced the likelihood of admission to UC Berkeley. Teranishi et al. (2004) took a broader approach by examining college access across public systems in California (community colleges, California State University, and the University of California) and found that access to different sectors of public higher education was associated with racial segregation and inequality in the state's public high schools. Martin et al. conducted a similar study focusing on how the differences in the socioeconomic status of different secondary schools in California correlated with postsecondary outcomes. These studies identify the extent to which disparities in access and outcomes in California are correlated with school demographics and the relative levels of segregation that exist in California's public high schools.

The premise for the current study builds on previous research by providing new perspectives for understanding problems related to racially segregated schools and postsecondary opportunities. Whereas previous studies have focused on factors associated with attending any public college or university in the state of California (Teranishi et al., 2004), the University of California as a whole (Martin et al., 2005), or a specific campus within the UC system3 (Chang, 2000), our research advances understandings of the association between high school segregation and opportunities for attending individual campuses within the most selective sector of public higher education in the state: the University of California. More specifically, this study examines the extent to which there is unevenness in the rate at which individual UC campuses enroll first-time freshmen from high schools, which varies by racial composition. Thus, this study examines the relationship between the racial composition of California's public high schools and the likelihood that a school is a source of first-time freshmen (FTF) at each UC campus. By studying the state of California, and specifically looking at the composition of freshmen at individual campuses within the UC system, we identify policies and practices that states can target to improve educational opportunities and outcomes.

Three research questions guide this study: (1) In what ways, if any, are the racial compositions of high schools different for FTF at different UC campuses? (2) How are the racial compositions of high schools that are the source of underrepresented minorities (URMs) at each UC campus different, if at all, from the racial compositions of high schools that are the source of White FTF? (3) Do the racial compositions of public high schools in California have an independent net effect on the likelihood that a school will send graduates to the University of California and individual UC campuses, controlling for other school characteristics?

This study examines the extent to which access to the University of California and/or its individual campuses is a systemic problem driven by gaps between state policies, the demographic context, and the disparate distribution of resources and opportunities for higher education. Although this study specifically examines California, it also offers implications for other states that struggle to diversify their public institutions in light of a segregated K-12 environment and their higher education policy environments. Thus, to explain how persistent segregation of public schools has affected the freshmen enrollment of students, which varies by race/ethnicity in public higher education, we place this issue in a broader context of state and higher education policy and the social conditions of public K-12 education.

BACKGROUND

Prior to the U.S. Supreme Court's historic Brown decision, racially divided schools not only existed but also were sanctioned by law. Although some believe that racial segregation was resolved in 1954, many regions of the country presently exhibit a resegregation of housing and education across racial and ethnic lines (Orfield & Lee, 2006; Orfield & Yun, 1999). Today, de facto segregation maintains racially isolated public schools throughout America. Nationally, Black and Latino students, on average, attend schools where they represent more than 50% of the student population. White students are even more racially isolated because they are likely to enroll in schools where 78% of the students are also White (Orfield & Lee).

The implications of racial segregation in schools are well documented in the secondary and postsecondary education literature. Racially isolated schools harm all students but have a greater negative effect on underrepresented minority students because these students are more likely to attend schools that lack important educational resources (Mickelson & Heath, 1999), such as access to qualified and experienced teachers, adequate facilities and supplies, and a college preparatory curriculum (Orfield & Lee, 2006). Despite the disadvantages of racial segregation, students in schools with higher proportions of minority students have higher expectations for degree attainment than those in majority-White schools (Frost, 2007). Yet, the gap in educational resources that are racially divided translates into differences in subsequent postsecondary educational and employment opportunities (Teranishi et al., 2004). The recent U.S. Supreme Court decisions to deny voluntary desegregation suggest that educational inequities will likely be preserved, if not further exacerbated.

California is an ideal case for examining the challenges that many other racially diverse states face related to the shifting landscape of racial segregation in public primary and secondary schools (Education Trust-West, 2004; Harris, 2004; Oakes, Rogers, Silver, & Goode, 2004). Public schools in the state are failing to provide students with qualified teachers, adequate facilities, sufficient textbooks, and a rigorous curriculum (Oakes et al.). Schools with inadequate instructional materials and underqualified teachers often assign faculty without the necessary training to teach various courses, and these schools have high teacher turnover (Harris). Further, many teachers in segregated schools reported that their schools were "overcrowded" and "unsafe" (Harris). Although these are concerns for all students, they are particularly distressing to Black and Latino students, who are disproportionately enrolled in schools with few qualified teachers (Education Trust-West; Oakes et al.), the fewest resources (Teranishi et al., 2004), and the fewest opportunities to complete the required course sequence (Education Trust-West; Oakes et al.). Oakes and her colleagues further argued that California schools "do not even pass the century old requirement of Plessy v. Ferguson" (p. 10), thus reflecting a system of schools that are separate and unequal.

Inadequate college preparation, particularly for students of color, has implications for postsecondary opportunities and outcomes. Researchers at the Education Trust-West (2004) have found that California ranks 49th in the United States according to the percentage of high school seniors who immediately enrolled in college, and 46th in the percentage of the college-age population earning a baccalaureate degree (Brown et al., 2006). The problems that exist in the K-12 sector are more pronounced by rigid higher education policies that govern the ways that public colleges and universities admit students. Public higher education in California is governed by a master plan that guarantees admission for state residents to one of three systems: University of California, California State University, and community colleges. The rigidity of the master plan is most pronounced in how the state is mandated to sort aspiring college students based on specific academic criteria. For example, public 4-year colleges in California require a 15-course sequence in high school for admission. However, too few high schools provide students with the opportunity to take these rigorous courses, with some claiming that California is offering "20th century diplomas in a 21st century economy" (Education Trust-West, 2004, p. 12).

Even among those students who are able to meet the rigid eligibility requirements to attend the most selective sector of public higher education in the state - the University of California - students are not guaranteed admission at any specific campus. As a result, students may apply to a more selective institution within the UC system, like University of California, Berkeley (UCB) or the University of California, Los Angeles (UCLA) but can be diverted to campuses that are less selective, like University of California, Riverside (UCR) or University of California, Santa Cruz (UCSC). Therefore, because admission to UC is based on a strict definition of minimum eligibility, and eligibility does not guarantee admission to the campus of choice, students' preparation and their ability to be competitive take on increasing importance. Persistent disparities within primary and secondary schools relating to underpreparation, however, are often overlooked when individual UC campuses consider the eligibility and competitiveness of their applicants. In other words, when campuses make decisions about any particular applicant, it is difficult to account for the extent to which their academic qualifications are affected by inequitable opportunities and limited access to the desired curricula and resources. This is an inherent flaw in the system because by design, applicants who cannot compete against the pool of other applicants against whom they are being considered must be relegated to a less selective UC, even if they meet UC's minimum eligibility requirements. If an applicant does not meet minimum eligibility requirements, he or she is sent to another sector of public higher education (e.g., CSUs or community colleges). Consequently, Black and Latino students in California are disproportionately represented in less selective colleges and universities of the same system (Allen et al., 2002).

We assert that the examination of access to higher education needs to be considered in a broader context of demographic changes, inequitable distribution of resources and opportunities in secondary schools, and the federal and state policy environment, which independently and collectively have important implications. None of these issues is unique to California. For instance, the student composition of many U.S. public high schools, particularly in urban metropolitan areas, increasingly and disproportionately comprises students of color (U.S. Department of Education, 2007), along with considerable inequities in school resources and facilities that continue to plague the effort of public schools to promote student learning and achievement. A deeper examination of both the ways in which diverse student populations from segregated high schools access higher education, and the types of institutions they have access to is critical to understanding educational inequality in higher education.

DATA SOURCE AND METHODOLOGY

This study was designed to examine the extent to which racial segregation of public high schools in California is correlated with disparate access for high school graduates to the individual campuses within the UC system. To accomplish this, we explored the high school source of first-time, full-time freshmen for 8 out of 9 individual UC campuses4 for the 2000-2001 academic year. As schools across the United States are becoming increasingly resegregated (Orfield & Lee, 2006), it is appropriate to continue to examine the relationship between the racial and ethnic composition of schools, and patterns of access, participation, and outcomes related to higher education.

DATA SOURCE

This study used data from the California Department of Education (CDE), California Postsecondary Education Commission (CPEC), and the University of California Corporate Student database, which contains information on high schools related to their enrollments, graduates, and college-going rates to public institutions in California (UCs, CSUs, and community colleges). Although these data sets have a wealth of information that is useful for studying the state of schools in California, it is important to note that the data were only available at the school level, which resulted in schools as the unit of analysis. Despite limitations associated with a single level of analysis, as opposed to multilevel analysis, this study is able to build on a number of other studies that demonstrate the significance of the relationship between school composition and school performance outcomes in education policy (Allen et al., 2002; Chang, 2000; Martin et al., 2005; Orfield & Lee, 2006; Teranishi et al., 2004). It is also useful for examining policies related to admissions, such as how school demographics factor into admission decisions in selective admissions, and more formal policies, such as the University of California's Eligibility in a Local Context (ELC) or Texas's 10 Percent Plan.

For the purposes of this article, we included only comprehensive public high schools, which resulted in the exclusion of public secondary schools that were magnet, charter, and continuation schools.5 Also not included in this study are comprehensive private high schools because of a lack of data comparable with the public school data used in this study.6 After controlling for these parameters, the data set contained information on 823 comprehensive public high schools in California.

MEASURES AND ANALYSIS PROCEDURE

The database includes measurements of school demographics and student outcomes. School demographic variables from CDE included measures of school enrollment (total enrollment, enrollment by grade level, and enrollment by race), student demographics (number of students eligible for the federal Free or Reduced Meal (FRM) program, average parental education, limited-English proficiency (LEP), locale (urban, suburban, and rural), and graduates (number of graduates by race). CDE and CPEC student outcome variables include the number and percentage of graduates who were eligible to attend, who applied to, or who attended a UC campus. Data from the University of California Office of the President (UCOP), which were merged with the data from the CDE database, include information on the number of first-time, full-time freshmen by race who attended each UC campus.

Two stages of data analysis were employed. First, we conducted descriptive and bivariate correlation to examine the relationship between the racial demographics of high schools and attendance rates of high school graduates, by race, to the University of California and individual UC campuses. The dependent variable in the univariate and bivariate analysis measured the number and percent of high school graduates from each high school by race who attended the UC system or each UC campus in 2000. We report results of the descriptive analysis for two cohorts of schools that varied by racial composition; the analysis was operationalized using Orfield and Lee's (2006) definition of a racial majority (50%-100% White or minority). The 50% threshold was only used for reporting descriptive analysis and was not used in the multivariate analysis. Based on these criteria, we ended up with 373 schools that were predominantly White in 2000, the largest number of schools with a majority concentration of any single racial group. In the same year, California had 309 high schools that comprised predominantly URM students. These cohorts of schools provided the foundation for subsequent analysis on attendance rates at UC campuses for high schools that varied in their racial composition. Schools that comprised predominantly Asian American and Pacific Islander students were excluded from the analysis because there were only 19 such schools in the state,7 which did not allow for statistically sound comparative analysis with the large numbers of White (n = 373) and URM (n = 309) schools.

Following the univariate and bivariate analyses, we conducted a series of multivariate analyses to examine the extent to which the racial composition of high schools has an independent effect on access to individual UC campuses, controlling for some key high school characteristics such as the socioeconomic composition of schools (% of students eligible for the federal FRM program), availability and use of Advanced Placement (AP) courses (% of total enrollment taking an AP course), availability of credentialed teachers (% of teachers with an emergency credential), and school locale (urban, suburban, or rural). These high school characteristics have also been found to be predictors of college attendance (see Martin et al., 2005; Kane, 1998; McDonough, 1997). The multivariate analyses consisted of 18 regressions, 2 for each of the 8 UC campuses in the study, and 2 for the entire UC system. The dependent variable is a continuous variable and measures the proportion of high school graduates who attended each UC campus, as well as an aggregate of the proportion of graduates who attended a school in the UC system.

The independent variables were divided into two blocks. The first block, which was applied to all the regressions, consisted of control variables: average parental education level, average proportion of English language proficiency, and school locale. The second block consisted of a variable that measures the racial composition of high schools. Half of the regressions for all samples included a measurement of the proportion of the school enrollment consisting of White students, and the other half of the regressions included an independent variable that measured the proportion of the school enrollment consisting of URM students. We opted to measure the effect of White concentration and URM concentration in separate regressions for two reasons. First, we did not want to include both variables in the same model because of concerns of a high degree of multicollinearity between these two variables. Second, having both independent variables in separate regressions measures the relative effect of each of them, as opposed to assuming that the directional effect found in one means the opposite effect in another. All independent variables were force-entered in the same order, with the exception of the two school racial composition measures.

LIMITATIONS

There are important limitations that are due to the data sources and variables included in this study. Because the unit of analysis is at the school level, we were not able to make inferences about the behavior of individual students. Additionally, we were not able to conduct multilevel analysis, which could identify the net effect of high school characteristics while controlling for individual characteristics. Accordingly, readers should be cautious in their interpretation of our results and consider that we are not assessing individual students; instead, we are focusing on the broader institutional enrollment patterns between California's public high schools and individual campuses within the UC system.

There are also important parameters to our data sample. The number of FTF included in this study is lower than what was reported by campuses because our data only include those students who originated from the 823 comprehensive public high schools included in our sample. Thus, excluded from our analysis were FTF who originated from in-state private high schools, as well as students who attended high school out of state (domestically and internationally). Another limitation is related to our dependent variable. This study only examined California high school graduates who enrolled at a UC campus, rather than studying students who were eligible for, who applied to, or who were admitted to the University of California.8 Each of these successive stages has important distinctions that are important to consider. These limitations do not make our findings less important, but the results should be considered with this in mind.

RESULTS

The results of this study are organized in the following manner. First, we provide profiles for California public high schools, which we grouped by racial and ethnic concentration to serve as the foundation for the subsequent analyses. Then, we compared the composition of FTF for the UC system and FTF for individual UC campuses both for representation from high schools that predominantly comprise URM students and from high schools that are predominantly White. This analysis was done separately for all students, for URM students, and for White students. Finally, we present results for the multivariate analysis, which identifies the independent effect of racial composition of schools on the composition of FTF to the UC system and individual UC campuses. The results are followed by a discussion of the results, the conclusion, and implications for policy and future research.

DEMOGRAPHIC OVERVIEW OF CALIFORNIA PUBLIC HIGH SCHOOLS

The 823 comprehensive public high schools included in this study were distributed across urban (28.4%), suburban (43.5 %), and rural (28.1%) regions of the state. The average rate of eligibility for the FRM program, a common measure of social class, was 11.9 % (see Table 1). The average total enrollment per school across all high schools was 1,852 students (see Table 2). These figures were quite different when compared across schools that varied by racial composition. The following analysis by levels of racial segregation in California's public high schools is important, considering that 84% of these schools have either a majority-White or majority-URM student population.

Table 1. Selected Characteristics of Segregated High Schools, 2000

All High Schools

(n = 823) URM

Schools

(n = 309) White

Schools

(n = 373)

Limited English proficiency 13.7% 21.4% 6.7%

Free meal eligibility 11.9% 18.5% 5.9%

Average parental ed levela 2.8 2.1 3.4

Urban locale 234 125 69

(28.4%) (40.5%) (18.5%)

Suburban locale 358 90 221

(43.5%) (29.1%) (59.2%)

Rural locale 231 94 83

(28.1%) (30.4%) (22.3%)

a 1 = some high school; 2 = high school diploma; 3 = some college; 4 = college degree; 5 = graduate/professional degree.

We found that 309 high schools had a majority (50% or higher) of their school population consisting of Black and/or Latino students (see Table 2). We identified these schools as URM schools. Slightly more than 60% of all Latinos and 48.1% of all Blacks in the state attended URM schools, compared with 29.7% of Asian American and Pacific Islander (AAPI) students and 20.8% of White students. URM schools had their largest concentration in urban areas (40.5%), with lower representation in the suburban (29.1%) and rural (30.4%) regions of the state. The average total enrollment for URM schools was 1,941, which was higher than the average total enrollment of all comprehensive high schools in the state.

Table 2. Total Enrollment by Race and School Segregation, 2000

All High Schools

(n = 823) URM

Schools

(n = 309) White

Schools

(n = 373)

Native American 12,840 3,979 6,462

(30.9%) (50.3%)

Asian American 147,425 43,779 76,887

(29.7%) (52.2%)

Latino 575,920 348,194 125,471

(60.5%) (21.8%)

Black 120,227 57,777 40,629

(48.1%) (33.8%)

White 607,753 126,563 401,902

(20.8%) (66.1%)

Other 59,784 19,536 30,248

(32.7%) (50.6%)

Total Enrollment 1,523,949 599,828 681,599

(39.4%) (44.7%)

A total of 373 had a White majority, which was the largest number of schools with a majority concentration of any single race. Sixty-six percent of all White students attended White-majority schools, which was a higher proportion than the representation of Black and Latino students in URM schools. The proportion of Blacks and Latinos who attended White schools (33.8% and 21.8%, respectively) was higher than the proportion of Whites who attended URM schools (20.8%). More than half (52.2%) of AAPI students in California were enrolled in predominantly White high schools. White-majority schools were mostly concentrated in suburban (59.2%) and rural (22.3%) areas, and their lowest representation was in urban neighborhoods (18.5%). These patterns reflect the propensity of Whites to reside in suburban communities within the state (Clark, 1998). White schools also had a lower average enrollment (1,827) than URM schools (1,941).

DISTRIBUTION OF HIGH SCHOOL GRADUATES IN CALIFORNIA HIGHER EDUCATION

Among California public high school graduates in 2000, 54% enrolled in California public higher education institutions; however, there was great unevenness in the proportion who attended community colleges (33.4%), CSUs (13.1%), and UCs (7.6%). Within segments, there are also important disparities across race and ethnicity that should be noted. A greater proportion of AAPI (20.3%) and White graduates (12.4%), as compared with Latino (4.5%) and Black (3.4%) high school graduates, attended UC campuses. A higher proportion of AAPI high school graduates (16.1%) also attended CSU campuses as compared with the similar CSU attendance rates of other racial and ethnic groups, including Latino (10.6%), Black (9.7%), and White (9.5%) California high school graduates. Latino (49.5%) and Black (34.1%) high school graduates had their greatest representation at community colleges, which was much higher than for AAPIs (27.6%) and Whites (28.3%). Thus, the distribution of public high school graduates in California varies greatly across different sectors of higher education in the state, with enrollment of FTF within segments also varying by race.

DISTRIBUTION OF PUBLIC HIGH SCHOOL GRADUATES IN THE UNIVERSITY OF CALIFORNIA

The following analysis looks specifically within the UC system, with a particular focus on the relationship between the origin of FTF and the varying levels of racial segregation that exist across California public high schools. For example, although 44.7% of all high school graduates attended high schools with a White majority, graduates from these schools represented 65.3% of FTF in the UC system (see Figure 1). For URM schools, which consist of 43.4% of all high school graduates in the state, the share of FTF in the UC system was only 21.7%.

Figure 1. Percentage of high school graduates and the percentage of first-time freshmen (FTF) for White-majority and underrepresented minority-majority schools, 2000

Note. URM=underrepresented minority.

Trends in disparate access to the UC system by race are apparent, but it is also important to examine whether the source of FTF by race also varied depending on the racial composition of their high schools. We found that although 58.3% of Blacks and/or Latinos in the state graduated from URM schools, only 44.7% of URM FTF came from these schools (Figure 2). An interesting finding was that although Blacks and/or Latinos had a much smaller proportion of students in White schools (23.9%), their share of FTF in the UC system was disproportionately high, at 40.3%. Therefore, there was nearly an equal proportion of URM FTF from White schools as from URM schools even though URM students are most likely to attend URM schools and least likely to attend predominantly White high schools. A great majority of White students attending the UC system came from White schools (75.6%), and few came from URM schools (14.9%; see Figure 3). All students from White-majority schools, therefore, had a greater representation in UC FTF than students from URM schools, regardless of the race of the UC entrants.

Figure 2. Proportion of underrepresented minority (URM) graduates and URM first-time freshmen (FTF) by high school composition, 2000

click to enlarge

Figure 3. Proportion of White graduates and White first-time freshmen (FTF) by high school composition, 2000

click to enlarge

Note. URM=underrepresented minority

SOURCE OF FIRST-TIME FRESHMEN FOR INDIVIDUAL UC CAMPUSES

Racial and ethnic disparities exist across and within public higher education segments as a whole in the state, but we were particularly interested in the extent to which these patterns are similar or different for individual campuses within the UC system. For each campus, we examined trends in access for different racial groups and compared them across campuses for students attending schools that varied by different racial and social class compositions. The differences were considerable.

For example, although 44.7% of all high school graduates attended high schools with a White majority, graduates from these schools represent 75.0% of FTF at UC Berkeley, 73.9% at UC San Diego, and 71.8% at UC Santa Cruz (see Figure 4). Conversely, although 43.4% of all students attended predominantly URM schools, FTF at UC Berkeley (15.2%), UC San Diego (17.2%), and UC Santa Cruz (18.8%) were extremely low among graduates of URM schools. UC Riverside (33.3%) and UC Irvine (27.1%) had the highest proportion of their FTF from schools that were predominantly URM. Graduates of URM schools were thus underrepresented at all UC campuses.

Figure 4. First-time freshmen (FTF) for different campuses by racial composition of high schools

click to enlarge

Note. URM=underrepresented minority.

The relationship between levels of racial segregation of high schools and the likelihood of graduates attending individual UC campuses was statistically significant at the .05 confidence level for all campuses except for UCLA and UC Davis (see Table 3). For graduates from URM schools, there was a negative relationship with attending UC Berkeley (-.24), UC Santa Barbara (-.23), UC Santa Cruz, (-.18), and UC San Diego (-.09), and a positive relationship with attending UC Riverside (.15) and UC Irvine (.09). There was a statistically significant positive relationship between graduates of predominantly White high schools and the likelihood of attending UC Berkeley (.22), UC Santa Barbara (.18), UC Santa Cruz (.12), and UC San Diego (.07), and a significant negative relationship with graduates from White high schools attending UC Riverside (-.11) and UC Irvine (-.10).

Table 3. Correlations Between Institutional Enrollment and Racial Composition of Schools

click to enlarge

Note. URM=underrepresented minority.

SOURCE OF URM FIRST-TIME FRESHMEN FOR INDIVIDUAL UC CAMPUSES

Although there were disparities in access to UC campuses for all graduates from different segregated schools, the discrepancies disproportionately impacted URM students in the state because they were more likely to attend URM schools than White students. For example, 63.3% of all Latino and/or Black graduates attended a URM school, yet at UC Berkeley, only 33.6% of URM FTF originated from URM schools (Figure 5); rather, nearly half (48.9%) of all URM FTF at Berkeley came from White schools. UC San Diego had similar trends, with 52.6% of URM students coming from White high schools. The campuses with the highest proportion of URM students who came from URM schools were UC Riverside (54.6%), UCLA (53.4%), and UC Irvine (49.6%). We believe that the higher rate of URM FTF from URM schools in these institutions is a reflection of the schools from which they draw their students, which are mainly located in Southern California. Los Angeles County, for example, has the highest concentration of racially segregated schools in the state and one of the highest concentrations of racially segregated schools in the country, which results in few opportunities for URM students to attend, and be admitted from, predominantly White high schools.

Figure 5. Underrepresented minority (URM) first-time freshmen (FTF) at different campuses by racial composition of high schools

click to enlarge

Note. URM=underrepresented minority.

SOURCE OF WHITE FIRST-TIME FRESHMEN FOR INDIVIDUAL UC CAMPUSES

The source of White FTF was also quite interesting. More than 66% of White high school graduates in California came from predominantly White high schools, which is much greater than the 20.8% who graduated from URM schools. For every UC campus, the proportion of White FTF from White high schools was greater than the proportion of White graduates from White high schools. UC Santa Cruz has the largest proportion of White FTF from White high schools (81.3%), followed by UC Davis (79.3%), UC Berkeley (79.0%), and UC San Diego (78.4%). Among the Whites who graduated from URM schools, their likelihood of attending some of the UC campuses was quite low. This was particularly true for White FTF at UC Santa Cruz (12.3%), UC Berkeley (12.4%), UC Davis (12.9%), UC San Diego (13.7%), and UC Santa Barbara (14.0%). This trend is similar to what was found among URM students graduating from URM schools.

Figure 6. White first-time freshmen at different campuses by racial composition of high schools

click to enlarge

Note. URM=underrepresented minority.

MULTIVARIATE ANALYSIS

Multivariate analyses tested for the independent effect of the racial composition of high schools sending graduates to the UC system and individual UC campuses, controlling for other school characteristics that have been found to be correlated with high college attendance (i.e., class composition of schools, availability and use of Advanced Placement courses, availability of credentialed teachers, and school locale; Tables 4 and 5). The systemwide regressions accounted for 54% of the variance in the model that included % enrollment White, and 56% of the variance in the model that included % of enrollment URM. The separate campus regressions had differences in variance explained by the model, particularly for the regressions that included % enrollment White (UC Davis had an R2 of .07, whereas UCLA had an R2 of .41), meaning that the model had a better fit for explaining the likelihood of FTF enrollment for some campuses than for others.

Regression results for the effect of URM enrollment on graduates attending the UC system and individual campuses confirmed patterns found in the bivariate analysis. After control variables were introduced, percent enrollment of URM was a significant negative predictor for attending the UC system (-.21), UC Davis (-.30), UC Berkeley (-.20), UC Santa Cruz (-.22), UC Santa Barbara (-.19), and UC San Diego (-.12). URM enrollment was a significant positive predictor for UC Riverside (.17) and statistically insignificant for UCLA and UC Irvine.

Results from the regression analyses that included % White enrollment was significant for 4 of the 8 campuses and was positively correlated with attending UC Santa Barbara (.31) and UC Santa Cruz (.21) and negatively correlated with attending UC Riverside (-.21) and UC Irvine (-.22). The lack of statistically significant results for the % White enrollment variable for the UC system and 4 of the UC campuses is most likely due to the explanatory power of the control variables, which tended to have a significant and higher effect size in those regressions compared with the regressions that measured for the effect of % URM enrollment.

Table 4. Predicting University of California (UC) Attendance (Systemwide and Individual Campuses) for California Public High School Graduates: The Impact of Underrepresented Minority (URM) Student Composition (With Control Variables)

UC System UCB . UCR . UCLA . UCD . UCI . UCSD . UCSB . UCSC .

b ? b ? b ? B ? b ? b ? b ? b ? b ?

Control Variables

% free meal eligible -0.023 -05* -0.004 -04 -0.009 -11** -0.002 -02 -0.002 -01 -0.002 -02 -0.003 -03 -0.002 -03 0.003 03

% of emergency credentials 0.013 02 0.008 05 0.024 19*** 0.007 04 -0.010 -05 0.010 07* -0.010 -06* -0.007 -05 -0.013 -09*

% of total enrollment taking AP 83.047 61*** 16.369 50*** 6.016 27*** 20.419 62*** 3.270 09* 11.665 43*** 12.439 45*** 10.053 39*** 3.166 12**

School in urban area (not rural) 2.148 15*** 0.369 11** 0.070 03 0.383 12** 0.694 17*** 0.241 09* 0.091 03 0.012 01 0.030 01

School in suburban area (not rural) 1.853 14*** 0.133 04 0.157 07* 0.200 06* 0.122 03 0.407 16*** 0.435 17*** 0.235 10** 0.015 01

School Racial Composition

% of enrollment URM -0.049 -21*** -0.011 -20*** 0.007 17*** 0.000 00 -0.020 -30*** -0.001 -01 -.006 -12*** -0.009 -19*** -0.010 -22***

Number of cases (schools) 823 823 823 823 823 823 823 823 823

(Adjusted R2) (56) (36) (13) (41) (12) (23) (35) (29) (09)

*p < .05. **p < .01. *** p < .001.

Table 5. Predicting University of California (UC) Attendance (Systemwide and Individual Campuses) for California Public High School Graduates:The Impact of White Student Composition (With Control Variables)

UC System UCB . UCR . UCLA . UCD . UCI . UCSD . UCSB . UCSC .

b ? b ? b ? b ? b ? b ? b ? b ? b ?

Control Variables

% free meal eligible -0.061 -13*** -0.015 -13*** -0.010 -12*** -0.005 -04 -0.016 -12** -0.011 -11** -0.007 -07* 0.002 02 0.003 03

% of emergency credentials -0.035 -05* -0.005 -03 0.024 18*** 0.004 02 -0.029 -13*** 0.001 01 -0.015 -10** -0.004 -03 -0.015 -10**

% of total enrollment taking AP 88.978 66*** 17.822 55*** 5.505 24*** 20.557 62*** 5.629 15*** 12.095 44*** 13.120 48*** 10.621 41*** 4.054 15***

School in urban area (not rural) 1.191 09** 0.073 02 -0.046 -02 0.262 08* 0.357 09 -0.093 -03 -0.001 00 0.266 10** 0.122 05

School in suburban area (not rural) 1.425 11*** -0.009 -01 0.077 04 0.132 04 -0.024 -01 0.221 09* 0.396 15*** 0.397 16*** 0.089 04

School Racial Composition

% of enrollment White 0.002 -01 -0.002 -03 -0.008 -21*** -0.004 -07 0.003 04 -0.010 -22*** .001 02 0.014 31*** 0.009 21***

Number of cases (schools) 823 823 823 823 823 823 823 823 823

(Adjusted R2) (54) (33) (14) (41) (07) (26) (34) (33) (09)

*p < .05. **p < .01. ***p < .001.

DISCUSSION

These findings provide evidence regarding the extent to which accessibility of the University of California or specific campuses within the UC system for public high school graduates is correlated with the racial composition of the high school from which they graduated, even after controlling for other school characteristics. Our analysis of the relationship between levels of racial segregation in schools and the likelihood that a school will enroll a graduate in the UC system shows that approximately two thirds (65.3%) of FTF originated from White high schools even though these schools comprised less than half (44.7%) of all graduates of public high schools in the state. Conversely, the number of FTF who originate from URM schools is disproportionately low (22.1%) considering their equal representation of the proportion of high school graduates in the state (43.4%).

The relative disadvantage of attending a predominantly Black or Latino high school is a particular problem for URM students who wish to attend a UC campus because they are much more likely to attend these URM schools than Whites. In fact, although a low proportion of URM graduates come from White schools (23.9%), they make up a large proportion of the URM FTF in the UC system (40.3%). However, this does not mean that if URM students attend White schools, they are just as likely as Whites to enroll in the UC system; rather, the advantage of attending a White school was most evident for White students, and, to a lesser degree, Latino and Black students. More than three quarters (75.6%) of White FTF, compared with 66.1% of White graduates, originated from predominantly White high schools.

For individual UC campuses, the inequalities were exceptionally pronounced. Some campuses had a very large difference in the proportion of FTF who came from White schools versus URM schools. For example, 75.0% of UC Berkeley's students originated from White schools, compared with 17.2% who came from URM schools. This trend was also particularly significant for UC San Diego. This is quite compelling considering that these campuses are located in, or directly next to, the top five counties with the highest number of Black and Latino residents in the state (see Teranishi, 2006). UC Riverside and UCLA were much more likely to have their students originate from URM schools. These results were strengthened by the regression analysis, which confirmed the independent effect of levels of racial segregation on the likelihood that a student will attend the UC system or a particular campus within the system.

Accessibility to individual UC campuses for Black and Latino students also varied depending on whether they attended a URM or White high school. At 5 of the 8 campuses in this study (UCSD, UCB, UCSC, UCSB, and UCD), a greater proportion of URM FTF originated from White schools, as opposed to URM schools, where they were most concentrated. The 3 other campuses (UCI, UCLA, and UCR) had URM enrollment that came from URM schools. White students from White high schools had the largest share of FTF enrollment across the 8 campuses in this study. For 6 of the 8 campuses, more than 75% of the White FTF came from high schools that were predominantly White. For UC Santa Cruz, 81.3% of White FTF originated from a predominantly White high school. These findings exemplify the extent to which the UC system has inequitable access and opportunities for students that vary by race and high school origin.

CONCLUSION AND IMPLICATIONS

This study examined the structural elements of public secondary schools in California that resulted in embedded privileges and disadvantages related to access to the UC system and individual UC campuses. California is the most diverse state with the most diverse students in the nation. However, despite the diversity, neighborhood and school segregation is widespread (Massey & Denton, 1993; Portes & Rumbaut, 1996) and results in the concentration of students in high schools based on race (Orfield, 1993). This study provides a portrait of the ways in which racial inequalities in access to higher education are compounded by the opportunities that vary for students based on the levels of racial segregation that exist across public high schools.

The impact of the racial composition of feeder schools for first-time freshmen was not uniform across the 8 UC campuses in this study. Thus, although the University of California as a whole may appear to be making gains in providing opportunities for students of color-and this has been reasonable to the extent that is tolerated by its residents-when access to individual campuses for different groups from different high schools is teased out, the discrepancies are quite significant. Students from URM schools should have the same opportunities to enroll in the most selective campuses of the UC as they do the least selective.

The inequities that are particularly pronounced at some UC campuses (UC Berkeley and UC San Diego) mimic those found in many other selective universities throughout the nation. But this problem seems to be "hidden" in many studies behind the more general outcomes of the system. Perhaps the most significant question that this study raises is centered on the efficacy of a policy that simply holds some UC campuses accountable for goals of equity in the state, yet allows other UC campuses to exclude whole groups of students based on the high school they attended through no fault of their own. We argue that such educational policy and practice is neither effective nor equitable. In addition to recognizing the uneven distribution and poor representation throughout the UC system of FTF who originated from predominantly Black and Latino schools, it is also important to consider the implications of the extent to which FTF are arriving in the UC system from schools that are predominantly White, which is disproportionately the case for White FTF. Systemwide, only 15% of White FTF came from schools that were composed predominantly of another racial group. This further exemplifies the need for racial diversity in the UC system if White students in the UC system are ever going to get exposure to non-White students. These results speak to the importance of diversity as a compelling interest for higher education.

Researchers, policy makers, and educational leaders need to be aware of these patterns of inequitable disparities and opportunities if California is to maintain its longstanding commitment to providing educational excellence for its residents. Policies and practices need to be put in place to end the reproduction of inequalities that persist in the California educational pipeline. For example, UC leaders, particularly those from individual campuses where URM high school graduates' chances of attending are low, should expand and improve outreach to URM schools, in addition to changing how students from these schools are viewed in competitive admissions decisions. Specifically, there is a need for policy makers, UC leaders, and future research to examine admissions policies to understand the ways in which they inherently privilege one group (those from White schools) and disadvantage other groups (those from URM schools). In addition, it is quite clear that this problem needs to be addressed through collaboration by both researchers and policy makers in higher education and the K-12 system in order to address both ends of what should be an unsegmented pipeline.

Future research can build on this study in at least two ways. First, it would be fruitful to conduct a similar study using longitudinal student-level data. This data could be merged with the institution-level data through the use of multilevel analysis, which would be able to identify the net effect of high school characteristics while controlling for individual characteristics. Second, there is a need, particularly in California, to also consider the high school source and college-going rate of the Asian American and Pacific Islander population. This population has a sizeable representation in the state overall but has an even greater presence in public higher education. Future research should account for how the postsecondary outcomes of this population are associated with the racial composition of their schools and communities.

Notes

1. Parents Involved in Community Schools v. Seattle School District #1 and Meredith v. Jefferson County Board of Education.

2. Gratz et al. v. Bollinger et al. and Grutter v. Bollinger et al.

3. UC Berkeley and UCLA are of particular interest because of their selectivity and national prestige.

4. Currently, there are 10 UC campuses. However, the data used in this study are from the 2000-2001 academic year and do not include UC Merced, which opened in 2005. Our analysis also excluded UC San Francisco because it does not admit undergraduate students.

5. We also excluded county and/or district diploma programs such as home schooling and continuation, which are included among the 1,006 public high schools recognized by the CDE.

6. For a good study on the relationship between public and private high schools with UC admissions, see Martin et al. (2005).

7. Studies that specifically examine schools that serve AAPI students and are situated in AAPI ethnic enclaves within the state of California have been conducted by one of the authors (see Teranishi, 2003, 2004).

8. Martin et al. (2005) examined how students are "weeded out" at each of these stages. Wilbur (2006) specifically looked at differences in the yield rates of admitted students who chose to enroll at UC campuses. She noted that "while the offer of admission is an institutional decision, the choice to enroll at a particular college is entirely a student and family decision" (p. 2).

References

Allen, W. R., Bonous-Hammarth, M., & Teranishi, R. (2002). Stony the road we trod: The Black struggle for higher education in California. Los Angeles: CHOICES, UCLA.

Brown, M., Edley, C., Gandara, P., Hayashi, P., Kidder, W., Ledesma, M., et al. (2006). California at the crossroads: Confronting the looming threat to achievement, access, and equity at the University of California and beyond (Discussion Paper). Chief Justice Earl Warren Institute on Race, Ethnicity, and Diversity, University of California Berkeley School of Law.

Chang, M. J. (2000). The relationship of high school characteristics to the selection of undergraduate students for admission to the University of California-Berkeley. Journal of Negro Education, 69(1/2), 49-59.

Clark, W. A. (1998). The California cauldron. New York: Guilford Press.

Education Trust-West. (2004). Are California high schools ready for the 21st century? Oakland, CA: Author.

Frost, M. B. (2007) Texas students' college expectations: Does high school racial composition matter? Sociology of Education, 80, 43-66.

Harris, L. (2004). Report on the status of public school education in California: With special emphasis on the status of equality in public school education. Menlo Park, CA: The William and Flora Hewlett Foundation.

Hunt, D. (2005). Racing with race: Strategies for competitive college access for African Americans 2005 CAPAA findings. Los Angeles: Ralph J. Bunche Center for African American Studies.

Hurtado, S., Milem, J., Clayton-Pedersen, A. R., & Allen, W. R. (1999). Enacting diverse learning environments: Improving the climate for racial/ethnic diversity in higher education. Washington, DC: Graduate School of Education and Human Development, The George Washington University.

Kane, T. J. (1998). Racial and ethnic preferences in college admissions. In M. Phillips (Ed.), The Black-White test score gap (pp. 431-456). Washington, DC: Brookings Institution Press.

Martin, I., Karabel, J., & Jaquez, S. W. (2005). High school segregation and access to the University of California. Educational Policy, 19, 308-330.

Massey, D. S., & Denton, N. A. (1993). American apartheid: Segregation and the making of the underclass. Cambridge, MA: Harvard University Press.

McDonough, P. M. (1997). Choosing colleges: How social class and schools structure opportunity. Albany: State University of New York Press.

Mickelson, R. A., & Heath, D. (1999). The effects of school and classroom racial composition on educational outcomes. Journal of Negro Education, 68, 566-586.

Oakes, J., Rogers, J., Silver, D., & Goode, J. (2004). Separate and unequal 50 years after Brown: California's racial "opportunity gap." Los Angeles: UCLA/IDEA Institute for Democracy, Education, and Access.

Orfield, G. (1993). The growth of segregation in American schools: Changing patterns of separation and poverty since 1968. Cambridge, MA: Harvard Project on School Desegregation to the National School Boards Association.

Orfield, G., & Lee, C. (2006). Racial transformation and the changing nature of segregation. Cambridge, MA: Harvard University.

Orfield, G., & Yun, J., T. (1999). Resegregation in American Schools. Cambridge, MA: Harvard University.

Portes, A., & Rumbaut, R. (1996). Immigrant American: A portrait (2nd ed.). Los Angeles: University of California Press.

Teranishi, R. (2003). California higher education: Is the whole better than its parts? Paper presented at the annual meeting for the Alliance for International Higher Education Policy (AIHEPS), Puebla, Mexico.

Teranishi, R. (2006). Black residential migration in California: Implications for higher education policy. Sacramento: Research and Policy Institute of California.

Teranishi, R., Allen, W. R., & Solorzano, D. G. (2004). Opportunity at the crossroads: Racial inequality, school segregation, and higher education in California. Teachers College Record, 106, 2224-2247.

U.S. Department of Education, National Center for Education Statistics. (2007). The condition of education 2007 (NCES 2007-064). Washington, DC: U.S. Government Printing Office.

Cite This Article as: Teachers College Record Volume 112 Number 6, 2010, p. 1575-1601

http://www.tcrecord.org ID Number: 15689, Date Accessed: 3/16/2012 1:12:21 PM