PTPA project evaluates its first four series: public TV's public affairs staples

Originally published in Current, June 2, 1997

Television viewing is naturally a subjective experience. Each viewer has likes and dislikes, and each station programmer has to accomodate a different mix of missions and audiences. Even those who admire the same program often disagree about why. So, how do TV professionals ever have a constructive conversation about program standards and content?

Last year, the Public Television Programmers Association took a shot at addressing this dilemma--by designing a systematic method to evaluate programs and get a dialogue going with producers about the real business of public broadcasting--programming.

The concept was modelled on the Public Radio Program Directors Association's "program user groups," which have critiqued public radio programs and published the results since the late 1980s. Mike Flaster, a PRPD founder who helped establish the user groups before adding TV responsibilities to his job at KPBS-FM/TV in San Diego, proposed that the newer TV programmers' group develop a similar methodology.

"There were a lot of different ways that programs were evaluated, but the people who were most capable of doing these evaluations needed their own instrument and agenda and criteria," said Flaster in an interview last year. "We wanted to be the advocate for the audience, which is the job of programmers." He suggested a process like the one used by PRPD.

Founded as a professional association for programmers to share knowledge, PTPA had never "used that expertise to help producers and help the programs that we air every day," said Judith LeRoy, codirector of TRAC Media Services, which administers PTPA. "We're keeping the posture of being helpful to producers."

At the start of the process, a cross-section of PTPA members receives a lengthy questionnaire with which they evaluate one or more specific telecasts of a chosen program. After the results are collated, PTPA shares them with the show's producers, who respond to the evaluations during PTPA meetings.

Until recently, PTPA didn't release or publish the critique summaries, as its radio counterparts do. Earlier this year, Current asked PTPA's executive officers to open up discussion of evaluated programs. PTPA released summaries of the news and public affairs programs it had critiqued so far: Frontline, The NewsHour with Jim Lehrer, Wall Street Week, and Washington Week in Review.

In each PTPA critique, programmers assessed several aspects of program quality: content, program purpose, personalities/talent/guests, technical quality, and consistency, among them. "Effectiveness" rates how well shows perform for stations in generating audiences, pledge dollars, or goodwill from their communities. The summaries also included selected remarks by unnamed individual programmers.

The most consistent theme in the critiques, by far, was the programs' limited "engagingness," (story, page 1), but there were many pointed and often contradictory remarks by individual programmers about other matters, such as the skills of hosts and correspondents. Programmers also revealed concerns about viewing levels and demographics.

"No one under 40 would ever watch (that's not necessarily bad for the program, but it is bad for the system)," wrote one reviewer of Washington Week. "This older audience is marginally beneficial to our station--they watch a lot of public TV already."

Other highlights:

• Frontline: Of all the PBS series critiqued so far, Frontline's "Navy Blues" received highest marks in its evaluation, drawing a 3.9 average score on a scale of 1 to 5. Ratings in the middle range of the score are labelled "average" for 3, and "superior" for 4.

  The Frontline program screened by PTPA was an examination of the Navy in the aftermath of the Tailhook sex scandal--a report that received kudos from the programmers, but its topic is not typical of all programs in the series.

  "Navy Blues" was "representative of what Frontline does best--investigative, in-depth and somewhat topical programs," wrote one reviewer. "But Frontline has two distinct faces. One is the long-form, rather self-indulgent obscure topic; the other falls into the 'news' realm. I prefer programs of this nature and so, apparently, does the audience."

  "Our viewers tune in based on the topic, not weekly," commented another. "Frontline needs to decide what it does best and create a tradition of excellence that viewers can depend on. It should be a series people don't want to miss."

  Also rated highly for enhancing PTV's image, Frontline's "Navy Blues" earned strong scores for program content, engagingness, and "presence"--the voices and people used to tell the story.

  "We believe that Frontline gives us the best, the very best of what public TV is all about," offered one programmer.

• The NewsHour with Jim Lehrer: Like Frontline, The NewsHour was given numerical scores for overall performance and various qualities. Two episodes from January earned a 3.3 ranking overall, about the same as the 3.2 for an episode from last year.

  The NewsHour received its strongest scores for mission-related categories, such as providing perspective on national issues, enhancing public TV's image, and achieving critic/journalist/peer recognition.

  The program's score for "engagingness" was 2.4 on the 1-to-5 scale, compared to Frontline's 4.2.

• Wall Street Week: Summaries for Wall Street Week and Washington Week in Review did not give numerical rankings, but PTPA noted where they were strong and weak.

  WSW received its best reviews for "host expertise, confidence and recognition," PTPA summarized. "There was, however, a concern expressed by the programmers that facets of [host Louis] Rukeyser's on-air persona may hinder him from attracting a new audience. Concern about those 'facets' resulted in low ratings on 'attitude and demeanor.'"

  The Rukeyser series is "one of the few programs we run which looks at financial information; it has a strong personality and weekly consistency," commented one programmer. "It encourages viewer loyalty in ways that most of our programs can't."

• Washington Week in Review: The panel of journalists, like host Ken Bode, "got its highest scores in preparedness and general professionalism," reported PTPA. "They were also highly evaluated in credibility and cooperation for the program's benefit. The lowest evaluation was for panel diversity (sex and ethnic), followed by enthusiasm."

  Washington Week in Review "serves the PBS viewership foremost by putting the national news in perspective," wrote one reviewer. "We also recognize its value as being regarded highly by Washington politicians."