|
|
Tweet |
|
Writing Project Professional Development Continues to Yield Gains in Student Writing Achievement
Date: July 15, 2010
Summary: These research results demonstrate that professional development programs designed and delivered by NWP sites have a positive effect on the writing achievement of students across grade levels, schools, and contexts.
Download the Research Brief
Key Findings
- In 16 studies conducted in seven states, 103 of 112 comparisons show positive results in writing achievement favoring students in classrooms of NWP participants.
- Student results are strong and favorable in those aspects of writing that the NWP is best known for, such as development of ideas, organization, and stance.
- Students in Writing Project classrooms gained more often than their peers in the area of conventions as well, suggesting that basic skills also benefit from the NWP approach to teaching writing.
- In the overall or holistic measure, in every case the improvement of students taught by teachers who participated in NWP programs exceeded that of students whose teachers were not participants.
Figure 1—Summary of Student Writing Performance
Click to enlarge chart.
In overall quality of writing (i.e., the holistic score), results consistently favor the NWP students in every single study. Click to enlarge chart.
Figure 2—Comparison of Student Writing Change
Click to enlarge chart.
In each case, program students show growth while comparison students often show little or no growth in their writing performance and, on occasion, show a decline in writing performance. Click to enlarge chart.
The Studies
All of the studies employed pre and post measures of student writing, comparing the performance of students whose teachers had participated in NWP programming to that of students whose teachers had not. Every study involved the direct assessment of student writing, independently scored at a national scoring conference where the Analytic Writing Continuum (NWP 2006, 2008) was applied with considerable reliability and technical rigor. The research proposal for each study—including research design, site capacity, and program—was subjected to an external review prior to acceptance into NWP's Local Site Research Initiative. Final reports of findings were then subjected to an external review prior to publication.
Context
Research teams in seven states conducted the 16 studies, including studies of 14 inservice programs ranging in length from 18 to 71 hours with a mean of 42 hours per teacher; a 10-hour family writing project; and a 100-hour invitational summer institute. Combined, the studies involved 141 schools, 409 teachers, and 5,408 students, covering a broad range of grade levels and diverse regions of the country: statewide, urban, suburban, and rural. Diverse economic, language, racial, and ethnic backgrounds are represented, with 6 studies focusing on student populations in which the majority were English language learners.
STATEWIDE STUDIES
Alabama—Teachers' Own Writing in Relation to Their Organization of the Classroom and to Student Achievement in Writing
Sunbelt Writing Project, Auburn University
A 2008 study examined teacher practices and student outcomes in secondary English classrooms of Writing Project teacher-consultants and in comparison classrooms. The study investigated whether teachers who write extensively themselves are more likely to view writing as an open-ended, rather than routine, task; whether this understanding would lead them to establish classroom environments where writing is taught as a non-routine composing process, rather than a prescribed, routine task; and whether these factors might predict student writing achievement. Teachers' survey reports were analyzed in relation to their students' performance on pre and post writing assessments.
California—Improving Students' Academic Writing
California Writing Project, University of California, Davis
This 2009 study investigated a program, Improving Students' Academic Writing (ISAW), developed by the California Writing Project network to provide sustained partnerships with teams of teachers from low-performing schools in both urban and rural areas. Professional development includes summer mini-institutes, school-year inservice and study groups, embedded assessment, and curriculum development. Year 1 findings indicate that students of ISAW teachers outperformed their comparison counterparts with significant differences in all 7 attributes of writing as measured by NWP's Analytic Writing Continuum assessment system and in all 18 attributes of writing as measured by the ISAW Writing Improvement Continuum assessment system.
Mississippi—The Effect of Mississippi Writing/Thinking Institute Professional Development on the Writing Achievement of Ninth-Graders
Mississippi Writing/Thinking Institute, Mississippi State University
A 2006 study examined the impact of Writing Project partnerships on the writing achievement of ninth-graders in two high schools with predominately African American student populations as compared with that of students in two comparison schools carefully matched for economic status, ethnicity, and prior performance of the student populations. Teachers in program schools participated in summer sessions, follow-up workshops, study groups, and classroom demonstrations on effective strategies for improving writing. Data included pre and post student writing samples, teacher interviews, and classroom observations. Analysis of student writing, classroom practices, and the relationships among practices and student outcomes formed the basis for program evaluation.
Missouri—Study of Missouri Literacy Academies
Missouri Writing Projects Network (MWPN), University of Missouri-Columbia and Missouri State University
A 2008 study examined the effects of the Missouri Writing Projects Network Literacy Academy model that sought to improve the teaching of writing in grades 6, 7, and 8 in "priority schools" (those not meeting Adequate Yearly Progress targets). To determine the effectiveness of this program, data from teachers involved with MWPN professional development were compared with data from teachers of similar grades who did not go through the program. Data sources included students' pre and post writing samples, semi-structured interviews, classroom observations, and teacher surveys. Program effectiveness was measured by student scores and an analysis of teachers' self-reported practices and beliefs.
URBAN STUDIES
Las Vegas—Through the Lens of the Family Writing Project: Impact on Student Writing and Teacher Practices
Southern Nevada Writing Project, University of Nevada, Las Vegas
A 2006 study investigated the effects of participation in the Family Writing Project (FWP), a program that brought together students, parents, and teachers from the highly diverse and mobile Las Vegas community to write about matters important to their lives. A second group of carefully matched students provided a comparative reference. Data including student writing samples; surveys; and interviews of teachers, students, and parents enabled researchers to study the effects of the FWP on students' writing achievement and attitudes about writing along with teachers' practices, including their approaches to teaching writing and their relationships with students and families.
New York City—The Impact of the New York City Writing Project: Teacher and Student Outcomes of a Professional Development Model for Improving the Teaching of Writing
New York City Writing Project (NYCWP), Lehman College
Research in 2006 focused on teacher and student outcomes resulting from NYCWP partnerships focused on improving writing instruction in six urban high schools. 2007 study investigated the impact of these partnerships on student writing at the same six high schools, where challenges include poverty, low student achievement, inexperienced teachers, and increasing pressures from high-stakes testing. The research examined how NYCWP professional development supported teachers' growth and affected student writing. Analysis of teachers' growth was based on interviews and surveys about instructional practices and attitudes. Students' growth was measured by pre and post assessments of student writing.
New York City—Improving Literacy Across the Curriculum: A Study of Instructional Development
New York City Writing Project, Lehman College
A 2008 study examined the degree to which weekly on-site professional development consultations and workshops affect schoolwide approaches to teaching writing, schoolwide instructional goals and policies, and student performance in writing. Researchers tracked the development of teachers at different levels of exposure, some participating in both consultations and workshops, some participating only in individual consultations or only in workshops, and some not having any contact. Students' writing growth was measured by pre and post assessments and compared across the three exposure groups. Analysis of teachers' and administrators' growth relied primarily on interviews and surveys regarding instructional practices, attitudes, and policy development.
St. Louis County—Increasing Student Achievement in Writing Through Teacher Inquiry: An Evaluation of Professional Development Impact
Gateway Writing Project, University of Missouri
A 2006 study examined the effects of a program that sought to develop a core group of teacher-leaders who could build and sustain a literacy-improvement model focused on the teaching of writing in grades 3, 4, and 5. A 2007 study examined the effects of a similar program at the middle and high school levels. The studies matched teachers and students on demographics and performance variables. Data sources included interviews, classroom observations, and student writing samples. Program effectiveness was measured by teachers' adoption of successful practices and by students' writing performance.
St. Louis County—Building Leadership for a Sustained, Districtwide Writing Improvement Program
Gateway Writing Project, University of Missouri
This 2009 study examined the effects of a three-year professional development program that sought to develop a core group of teacher-leaders who could build and sustain writing improvement programs at their schools. Program effectiveness was measured by teachers' classroom use of writing process and effective instructional strategies, as well as students' writing performance. Data included student writing, teacher interviews, and classroom observations. Comparison teachers, particularly by the third year of the study, had been influenced by ideas and workshop materials shared by their program colleagues.
URBAN/RURAL STUDIES
Ventura and Kern Counties, California—Evaluating IIMPaC: Teacher and Student Outcomes Through a Professional Development Program in the Teaching of Writing
South Coast Writing Project, University of California, Santa Barbara
A 2006 study examined teacher and student outcomes of a partnership focused on writing improvement in four elementary and middle schools serving high-poverty populations with substantial numbers of English language learners. A 2007 study examined the effects of a program focused on the teaching of writing in grades 3–8. Surveys, interviews, classroom observations, and teacher and student work were used to measure the effect of the professional development on teachers' classroom practice. A survey adapted from an established measure of writing apprehension (Daly and Miller 1975) examined the impact on student attitudes. Writing performance was measured by administering timed writing prompts in a pre/post fashion.
SUBURBAN STUDIES
South Carolina—Evaluating Project WRITE: Teacher and Student Outcomes of a Professional Development Program Focusing on Core Components of Writing Workshop and the Traits of Writing
Upstate Writing Project, Clemson University
This study, conducted over two school years—2008 and 2009—with two sets of program and comparison schools, documented the effects of a professional development program, Project WRITE (Write, Respond, Instruct, Talk, Evaluate): A Model for Changing Teaching Practice and Impacting Student Writing. Multiple pre and post qualitative indicators were collected to determine what impact the professional development had on third, fourth, and fifth grade teachers' practice and philosophy at the program schools. Quantitative measures were used to examine student writing performance on a pre and post prompted writing sample for all students in Year 1. In Year 2, these prompted writing measures were supplemented by pre and post measures of portfolio pieces written in class and revised over time.
SUBURBAN/RURAL STUDIES
Mississippi—Effects of NWP Teaching Strategies on Elementary Students' Writing
Mississippi Writing/Thinking Institute (MWTI), Mississippi State University
This 2007 study examined the impact of MWTI professional development on teachers' instructional strategies and on student writing outcomes in grades 3, 4, and 5 of a rural school, in comparison to outcomes in a carefully matched suburban school with similar demographics and similar test scores. The MWTI program included workshops, classroom demonstrations, and coaching on responding to student drafts. Data included student pre and post writing samples and teacher interviews. A continuum of implementation was used to determine the degree to which teachers implemented instructional strategies presented in NWP professional development. Further analysis sought to determine relationships between implementation of practices and student outcomes. The study also examined the correlation between scores on prominent and syntactic features in students' writing and their holistic writing assessment scores.
Results
On seven measures of writing performance tested across the 16 studies, students of NWP teachers outperformed their non-NWP counterparts in 103 of 112 contrasts. In 55% of these positive contrasts, the differences were so large as to be statistically significant. In eight cases, the comparison group outperformed the Writing Project group, but the differences were not significant. And in one case there was no difference between groups. These findings—the overwhelmingly positive results favoring NWP and the fact that in no case did the comparison group significantly outperform the students in NWP classes—confirm the effectiveness of NWP professional development.
Figure 1, Summary of Student Writing Performance (above), summarizes findings on the seven measures of writing performance. Upward-pointing triangles represent positive findings: students of NWP teachers outscored their counterparts who did not have NWP teachers. Solid triangles represent differences large enough to be considered statistically significant. Particularly strong results were demonstrated in the areas of content, structure, and stance: elements that address the quality of thought and the manner in which it is expressed. The circle represents equal gains in both groups. Downward-pointing triangles represent instances in which comparison groups outperformed the NWP groups; however, no significant differences were found. In overall quality of writing (i.e., the holistic score), results consistently favor the NWP students in every single study, with results in 8 of the 16 cases considered statistically significant.
Overall Student Change from Pre to Post Assessments
Figure 2, Comparison of Student Writing Change (above), illustrates the amount of change in writing performance exhibited by program and comparison students in each of the 16 studies, measured holistically. In each case, program students show growth while comparison students often show little or no growth in their writing performance and, on occasion, show a decline in writing performance.
Methodology
While every study was tailored to local contexts to enhance its validity, certain methodological elements were consistent across all studies:
Comparative Reference
All studies incorporated comparison groups selected for similarity to program groups in grade level, economic status, percentage of ELL students, type of community, ethnicity, policy contexts, and (where available) scores on previous measures of performance.
Pre and Post Writing Assessments
Student writing samples were taken from both program and comparison groups at the beginning of the school year and again at the end, after delivery of the professional development. Samples were analyzed for growth between pre and post assessments for program and comparison groups. Differences in growth between program and comparison groups were also analyzed.
Independent Scoring of Student Writing
To ensure technical rigor and credibility of data, scoring and data processing were conducted nationally and independently of local sites. All student writing samples were scored using a common evaluative framework, the NWP Analytic Writing Continuum (AWC). Scorer reliability (defined as inter-rater agreement) was 90% across all scoring categories.
Students, Teachers, and Schools
Represented by the 16 studies are 141 schools, 409 teachers, and 5,408 students, including public school students in third through twelfth grades; students receiving free or reduced-price lunch as well as those paying full price; students in urban, small city, and rural communities; and students of differing ethnicities. In 6 of the studies the majority of students were English language learners.
REFERENCES
Blau, Sheridan D., Rosemary H. Cabe, and Anne Whitney. 2006. Evaluating IIMPaC: Teacher and Student Outcomes Through a Professional Development Program in the Teaching of Writing. Berkeley, CA: National Writing Project.
Blau, Sheridan D., Rosemary H. Cabe, and Anne Whitney. 2007. Evaluating IIMPaC: Teacher and Student Outcomes Through a Professional Development Program in the Teaching of Writing. Berkeley, CA: National Writing Project.
Campos, Anne, and Roger Peach. 2006. The Impact of the New York City Writing Project: Teacher and Student Outcomes
of a Professional Development Model for Improving the Teaching of Writing. Berkeley, CA: National Writing Project.
Campos, Anne, and Roger Peach. 2007. The Impact of the New York City Writing Project: Teacher and Student Outcomes
of a Professional Development Model for Improving the Teaching of Writing. Berkeley, CA: National Writing Project.
Campos, Anne, and Roger Peach. 2008. Improving Literacy Across the Curriculum: A Study of Instructional Development. Berkeley, CA: National Writing Project.
Daly, J.A., and M.D. Miller. 1975. "The Empirical Development of an Instrument to Measure Writing Apprehension." Research in the Teaching of English 9: 242–249.
Kaminski, Rebecca, and Sarah Hunt-Barron. 2010. Evaluating Project WRITE: The Teacher and Student Outcomes of a Professional Development Program Focusing on Core Components of Writing Workshop and the Traits of Writing. Berkeley, CA: National Writing Project.
Lannin, Amy, and Keri Franklin. 2008. Study of Missouri Literacy Academies. Berkeley, CA: National Writing Project.
Marlink, Jayne, and Juliet Wahleithner. 2009. Improving Students' Academic Writing. Berkeley, CA: National Writing Project.
McKinney, Marilyn, Saralyn Lasley, and E. Michael Nussbaum. 2007. Through the Lens of the Family Writing Project: The Southern Nevada Writing Project's Impact on Student Writing and Teacher Practices. Berkeley, CA: National Writing Project.
National Writing Project. 2006, 2008. Analytic Writing Continuum. Berkeley, CA: National Writing Project.
National Writing Project. 2008, January. Local Site Research Initiative Report: Cohorts II and III 2004–2005,
2005–2006; Summary Report of National Results. Berkeley, CA: National Writing Project.Singer, Nancy Robb. 2009. Building Leadership for a Sustained, Districtwide Writing Improvement Program. Berkeley, CA: National Writing Project.
Singer, Nancy Robb, and Diane Scollay. 2006. Increasing Student Achievement in Writing Through Teacher Inquiry: An Evaluation of Professional Development Impact. Berkeley, CA: National Writing Project.
Swain, Sherry Seale, Richard L. Graves, and David Morse. 2006. The Effect of Mississippi Writing/Thinking Institute Professional Development on the Writing Achievement of Ninth-Graders. Berkeley, CA: National Writing Project.
Swain, Sherry Seale, Richard L. Graves, and David Morse. 2007. Effects of NWP Teaching Strategies on Elementary Students' Writing. Berkeley, CA: National Writing Project.
Whyte, Alyson. 2008. Teachers' Own Writing in Relation to Their Organization of the Classroom and to Student Achievement in Writing. Berkeley, CA: National Writing Project.
Download "Writing Project Professional Development Continues to Yield Gains in Student Writing Achievement"
