Current Online

Outreach: what impact does it have?
WGBH finds it boosts teachers' usage of programs

Originally published in Current, June 17, 1996
Commentary by Beth Kirsch

Ten years ago at WGBH, our only national outreach project was the Nova Teacher's Guide, and there was no person or department responsible for outreach beyond our local community. Today, national outreach at WGBH has become a $3 million-a-year enterprise employing more than 20 people, who carry out various outreach campaigns, conduct workshops around the country for educators, and produce a broad range of teaching materials for Nova, Where in the World Is Carmen Sandiego?, Masterpiece Theatre and many other public television programs.

The growth of outreach at WGBH, and at many other PBS stations, begs the question: does all this outreach actually have any impact—beyond attracting funds that in previous years might have been earmarked solely for television production?

Nearly everyone involved in outreach has a treasured collection of anecdotes that attest to the value and impact of what we do. But anecdotes aren't enough to convince funders, producers or even ourselves that outreach has an important place in public broadcasting. The good news is that during the past five years, independent evaluations we have commissioned to assess our outreach activities have demonstrated that:

The bad news, which I prefer to look at more as an ongoing challenge for WGBH and anyone else in the outreach biz, is that too many outreach projects are designed without measurable objectives, and not enough money is available to evaluate the impact of everything we do.

At WGBH, we've chosen to target most of our outreach activities toward educators—primarily K-12 teachers, but also child-care providers and leaders of community and youth-serving organizations. This focus has limited the kind of outreach projects we'll take on, but has also enabled us to develop a significant and loyal constituency.

Attracting broadcast audiences through outreach

Evaluations conducted for us by independent researchers have consistently shown that outreach activities, especially distribution of teacher's guides, encourage teachers to watch and use WGBH programs. For example, in a 1994 evaluation of teachers who receive our Nova guide, 91 percent reported that it enables them to use more programs than they might otherwise; 88 percent use the guide and programs together with their students.

An evaluation of the poster/guide we created for American Playhouse's "Fires in the Mirror" compared people who received our materials against a control group of teachers who were not mailed the poster. Despite strong publicity about the program in the New York Times and other major newspapers across the country (as well as in station program guides), among the people surveyed who knew about the show, 70 percent had heard about it from the poster. Only 19 percent of teachers who hadn't received the poster were familiar with the program, while 77 percent of poster recipients were. And 66 percent of recipients of the poster said it substantially raised their interest in watching the program.

Similarly, more than half of the teachers surveyed in an evaluation of our Carmen outreach activities decided to use the series in the classroom because they had received educational materials from WGBH. (Other reasons included their familiarity with the software, student excitement about the program, workshops we conducted, and recommendations from other teachers.) And 79 percent of teachers interviewed by phone as part of this evaluation said that free, high-quality materials make it very or extremely likely that they'll use television or video in the classroom.

Changes in learning, attitudes and behaviors

Building awareness about a new program is relatively simple; it's much harder for outreach to make a real difference in what people know, what they think about a topic, and how they go about their lives. Outreach projects can succeed in all three of these areas, if the goals are clearly—and specifically—defined up front, along with a plan at the outset for measuring whether the strategies for achieving these goals have succeeded.

For example, for the American Experience program "America and the Holocaust: Deceit and Indifference," a local outreach project we created sought to:

We established a partnership with an organization called Facing History and Ourselves, a national leader in bringing Holocaust education to schools, to conduct a half-day workshop for teachers from the Boston area. We also provided a teacher's guide and videocassette of the program for all participants. Our evaluation plan included a written survey that teachers filled out at the end of the workshop, along with follow-up phone interviews the following school year, to track how well their initial comments squared with their responses once they were back in their classrooms.

The workshop held in April 1995 was attended by about 100 teachers, and phone surveys were conducted with a subset of participants (more than 30) in January and February 1996. Both immediately after the workshop, and then 10 months later, teachers gave the workshop high ratings for increasing their knowledge. Two-thirds of the teachers taught the subject the following year (many more planned to cover the topic in the spring), and most had used the video and the guide.

On a much larger scale, our national "S.O.S.: Seek Out Science" outreach project for Discovering Women, a six-part series that aired last spring, sought to challenge young people's stereotypes about science and scientists, and to heighten interest in science among girls and young people of color. This ambitious outreach campaign, funded by the National Science Foundation, focused on creating opportunities for young people to meet female scientists in diverse occupations. The main project components were an activity guide for middle school teachers and youth leaders, museum events and exhibits, conference presentations, and mini-grants to PBS stations.

An extensive evaluation included:

Students overwhelmingly rated the interviews with female scientists as their favorite part of the project, because they "got to see what it's really like to be a scientist." As a result of the project, a significant number of students changed their definition of science and their views about who can become scientists. For example, before the project, students were likely to define science as "chemistry" or "experiments"; afterwards, they were more likely to describe science as "finding the answers to problems." Students' responses about who can be a scientist also changed—for example, from "anyone smart" to "anyone."

Before the project, students of color were significantly less likely than white students to consider the possibility of becoming a scientist. And, white students were more likely than students of color to have positive attitudes about science in their lives; participating in the project eliminated this gap. Girls' attitudes about women and people of color pursuing science careers, positive to begin with, became even more positive by the end of the project.

Teachers, too, rated the project highly. The vast majority of teachers who used the activities in the guide said it was very or extremely valuable in:

WGBH has also evaluated the impact of our local Ready to Learn activities, especially the use of the free books provided by the First Book organization and CPB. In phone surveys conducted last June with a subset of family day care providers participating in the project, 91 percent rated the content and ideas presented in our training as excellent. Nearly three-quarters of the providers are reading the free books with the children in their care (not just distributing them), and many are also carrying out a range of activities related to the books.

One of the most encouraging findings is that 75 percent of the family day care providers reported that the children's attitudes about reading have changed: they like to read more, and they ask to be read to more often. The project has affected parent behavior, too; 50 percent of the providers noted that parents now read with their children more than they used to.

The reach and use of teachers' guides

Not every teacher who receives a teacher's guide will use it—or even remember that they received it in the pile of magazines, direct-mail offers and memos stuffed into their mailboxes. Sometimes the featured television program doesn't interest them, the topic may not fit their curriculum, or the design doesn't attract their attention enough to get them to look through the material.

But national distribution of teachers' guides can lead thousands of teachers and millions of students to watch and use public television programs. For example, twice each year WGBH distributes 45,000 Nova Teacher's Guides nationwide. In an independent survey of 400 teachers, 85 percent reported that they received the guide, and of those, 72 percent used the lessons in the classroom, and 59 percent shared it with an average of four additional teachers. Overall, teachers reported that they used the guide with an average of 95 students. According to CPB's 1990-91 "Study of School Uses of Television in the Classroom," Nova is used by more than 35,000 teachers reaching nearly 3 million students, and is among the top five television programs (on PBS, cable or network TV) most frequently used in the classroom.

In evaluations of many of our educational materials, 75-95 percent of teachers rate them as very or extremely useful. At least 50-75 percent report that they plan to use them again the following school year. Components that they rate most highly include background information, pre-viewing and post-viewing discussion questions and activities, reproducible pages for students, and posters.

When outreach doesn't succeed

Evaluations also help pinpoint areas for improvement. Getting materials into the hands of teachers who will use them is a persistent problem, especially given the unreliability of bulk mail delivery and inconsistent routing of mail that makes it to the school building. We tried to avoid this problem by including teaching materials for Discovering Women in magazines published by the National Science Teachers Association, but evaluation showed that teachers' recall and use of this material was extremely low. We had also hoped the Discovering Women project would encourage thousands of kids across the country to create presentations about the female scientists they met, for display in museums; only about 1,000 kids carried out this part of the project, because of time constraints and other factors.

Finding out what went wrong with a project is as important as finding out what worked—it has helped us avoid making the same mistakes twice. The more evaluations we've carried out, the better we've become at establishing achievable, measurable goals at the outset of a project. And these evaluations have enabled us to see for ourselves whether outreach can have an impact. It can, and it should. Otherwise it's not worth the time or the money we're investing in it.

Beth Kirsch, director of educational outreach at WGBH, has overseen outreach activities for Nova, Masterpiece Theatre, Degrassi High, Where in the World Is Carmen Sandiego?, Discovering Women, Arthur, Ready to Learn and many other WGBH productions. The teachers' guides WGBH created for both Henry V and Middlemarch received Distinguished Achievement Awards for Excellence in Educational Journalism from the Educational Press Association; the Middlemarch guide was named Best of Category in 1995.

. To Current's home page

Web page posted Nov. 4, 2000
Current
The newspaper about public television and radio
in the United States
A service of Current Publishing Committee, Takoma Park, Md.
E-mail: webatcurrent.org
301-270-7240
Copyright 2000