• Print

    Science of education: Reformers put Memphis, Shelby County teachers to the test

    By Sarah Garland - The Hechinger Report -

    Rebecca Sellers, an eighth-grade English teacher at Lester School in Binghamton, looked wary as she walked into the teachers' lounge on a Monday last fall. The previous week, the school's assistant principal, Dr. Isaac Robinson, had dropped in, unannounced, to watch Sellers teach as part of Tennessee's new evaluation system.

    Now he was about to reveal her scores. As he fiddled with a computer connected to a projector, Robinson asked Sellers how she thought she did.

    "I'm not sure how I did, because I had to make some adjustments," said Sellers. Her students tend to do well on state tests, but the lesson hadn't gone as planned. Her eighth-graders had been stumped by a quick review exercise on pronouns. Sellers had taken an extra 15 minutes to go back over the material.

    "I had to meet the children where they were at that particular time," she added. "Do you remember the lesson?"

    Robinson, a transplant from Georgia in his second year at Lester, a high-poverty school in the Binghamton neighborhood where 97 percent of students are black, said he did. He had been taking notes on a new iPad provided by the school district.

    With a click, he displayed Sellers' scores on the projector: Mostly 2s, and even a 1, on a 5-point scale. Sellers, a 17-year veteran, was on track to losing tenure and possibly her job if she didn't score at least 3s in the future.

    "Let me read what a 3 looks like: 'Teacher communicates lesson objectives to students,'" said Robinson, reading from a chart on the screen. "I don't think that was done."

    Sellers rolled her eyes. "Well, if they didn't know what the focus was, they wouldn't know what they were supposed to do, right? And they did what they were supposed to do," she said.

    The discussion deteriorated from there. Forty minutes later, Robinson typed "off-task behavior interfered with instruction time" into a form on the computer screen. Sellers sat with her arms crossed, shaking her head.

    "I don't agree with this evaluation at all," she said. "I don't think it reflects the job I did."


    This fall, principals and assistant principals fanned out into thousands of Tennessee classrooms in an unprecedented effort to spend at least an hour annually observing and rating every teacher, guidance counselor, social worker and librarian in the state's public school system. Their goal: find teachers who are struggling, figure out what they are struggling with, and help them get better.

    In Memphis and Shelby County, anecdotal reports suggest most feedback sessions have not been as charged as the one between Sellers and Robinson.

    "We're not about 'gotcha,' " said Memphis City Schools Supt. Kriner Cash. "We're not about catching teachers being level 1 or level 2 and then trying to figure out ways to get them out of the profession."

    Yet the exchange at Lester highlights the challenges both Memphis and Shelby County schools face as they roll out their new evaluation systems and attempt to retrain the local teaching force.

    The effort is part of a sea of change in public education across the country, with Tennessee, whose students have long ranked near the bottom on national tests, at the forefront. Education reformers, including those in the Obama administration, have embraced the belief that great teaching is not an art, or, as Cash puts it, something "born in you." Rather, they see great teaching as a science-- something that can be taught and learned.

    To that end, states and districts, aided by hundreds of millions of federal and philanthropic dollars, are developing intensive evaluation systems meant to identify teachers who need help, and pinpoint with which skills they need help. Under a state law passed last spring, teachers must be formally observed at least four times a year, or six if they're new to the profession.

    A teacher's observation scores are supplemented by a "value-added" rating, which is calculated by determining whether a teacher's students made greater gains on standardized tests than statistical models would have predicted. But because value-added ratings don't come out until after the school year is over-- and because the majority of teachers don't teach subjects with annual standardized testing-- the revamped observations have become a major piece of the reform effort.

    "If you look at any teacher anywhere, they all think that they're great, and they're all working hard and they're trying," said David Stephens, assistant superintendent of curriculum and instruction in the Shelby County Schools. "Sometimes they may just not have enough knowledge, or some of the skills may be lacking. So if we do some things to help improve that, then I feel like we're headed in the right direction."

    The question is whether the new system can work where decades of other education reforms have not. Are observations accurately identifying struggling teachers? Are teachers learning from the feedback they receive? Are they finding resources to help themselves improve? And, most importantly, are students performing better as a result?

    "We have a need to identify our true underperformers. There are teachers that are just harmful to kids ... academically harmful," said Sandi Jacobs, vice president of the National Council on Teacher Quality, a Washington-based advocacy group. "But we have lots of teachers who aren't as good as they could be, and that is where the thrust of this work really is, the desire to maximize the teaching force."

    Both districts see the reforms as urgent, even though their student populations are very different. One third of students in Shelby County Schools are signed up for free- or reduced-price lunch, a measure of poverty, and nearly 40 percent are black, according to data from the state department of education. In Memphis, one of the poorest cities in the nation, 87 percent of students get subsidized meals, while 84 percent are black.

    Students in both Shelby and Memphis have made big gains on math tests in recent years, but only half of county students tested proficient or advanced on state math tests in 2011 and Memphis still ranks at the bottom in terms of proficiency.

    In reading, gains have been smaller for both districts. In Shelby, 57 percent of students were proficient in 2011, compared to just a quarter of students in Memphis, according to the state education department.

    Less than a year into the new system, the Memphis and Shelby County school districts have conducted nearly 10,000 observations of the nearly 10,500 teachers, librarians and other instructional staff in the districts. They already are compiling data and hearing reactions -- both positive and negative-- from teachers and principals.


    Many teachers and principals say the biggest change is the amount of time principals are spending in classrooms. Previously, teachers in Tennessee were evaluated once every five years.

    Under the new system, principals are required to spend from 60 to 90 minutes in a teacher's classroom annually, depending on a teacher's experience -- meaning for veteran teachers, principals must conduct four 15-minute observations over the course of the school year.

    In Memphis, however, district administrators have found principals are actually spending an average of 29 minutes doing each observation. Anecdotal reports in Shelby County suggest principals there are also going beyond state requirements. In addition, conferences between principals and teachers after an observation can run as long as an hour.

    Some principals and teachers have complained about the amount of time they spend doing evaluations, but others appreciate the shift. "It used to feel like I was learning things on my own. I was observed, but I didn't get targeted feedback like I do now," said Davida Smith-Keita, an English teacher at Manassas High School. "I think it's time well spent."

    Observers grade teachers on four different "domains," including planning, teaching and classroom environment. Each domain includes a list of standards, or indicators, such as using strategies to promote higher-level thinking skills or creating a respectful classroom culture. In the classroom of an above-average teacher, for example, students should come up with their own questions and be able to explain concepts to one another with the teacher's help. In contrast, a below-average teacher spends most of the lesson talking and only calls on volunteers, according to the standards.

    During post-observation conferences, teachers receive a score on a five-point scale. Level 1 and 2 are "below expectations." Level 3 is "meeting expectations." And levels 4 and 5 are "above expectations." Evaluators are supposed to point out areas in which teachers can improve and suggest how they might reach the next level.

    "The substantive conversation is really the big deal. We're already hearing from teachers that it's a very useful experience," said Irving Hamer, deputy superintendent of academic operations, technology and innovation for MCS.

    In response to complaints about time, the state recently decided to allow observers to focus on two domains at a time during half-hour sessions, so veteran teachers will only be observed twice a year and first-year teachers will be observed three times. Memphis, which began work on overhauling its teacher evaluation program two years ago and uses a different system than the rest of the state, is deciding whether to adopt the change as well.

    Some elements of the new system are similar to the old way, although the terminology has changed.

    Before, nearly every teacher received a satisfactory score. So far, new observation scores have also skewed positive, something that neither district is celebrating.

    In Memphis, more than 1,000 out of about 7,000 teachers, or about 14 percent, were rated "below expectations" in the first round of classroom observations this fall. By contrast, about 46 percent scored "below expectations" based on value-added student test scores last year. In Shelby County, evaluators have rated only about 6 percent of teachers at level 1 or 2 in their observations so far this year, compared to 25 percent who were rated level 1 or 2 based on last year's value-added test scores.

    Then there is the case of Rebecca Sellers, who received low marks during her two observations this year, but the top rating, 5, based on her students' test score progress last year.

    Making sure the value-added data and the observation ratings are in the same ballpark is an important test of accuracy for both measures, according to a study published last year by Measures of Effective Teaching, a research project in Memphis funded by the Bill & Melinda Gates Foundation. Principals are also being evaluated by district administrators, and part of their scores depend on whether the observation ratings they give teachers correlate with the value-added data that come out at the end of the year.

    If they don't match up, the system's usefulness and reliability could come into question, and it could lose credibility among educators.


    On a recent November morning just after announcements, Kay Obenchain, a teacher now in her seventh year, entered the principal's office at Millington Middle School. About two-thirds of the students at Millington qualify for free- or reduced-price lunch, and more than a third are black. Dr. Michael Lowe, Millington's principal for the past seven years, had observed Obenchain teach a sixth-grade math lesson earlier in the week.

    Lowe pulled out a legal pad with his notes.

    "How do you think the lesson went?" he asked.

    "The kids seemed to enjoy it," she said. "They learned the objective I set."

    At the end of the 12-minute conversation, Lowe revealed her scores: 4s and 5s. Obenchain looked relieved.

    Afterward, she praised the process: "I like it better than the old observation," she said. "This breaks it all down so we can see what a teacher 'above expectations' looks like."

    Obenchain's positive reaction to her observation is common, according to Laura Link, director of professional learning and development in Shelby County. "The ones who have been, over time, getting successful scores appreciate this model," she said.

    For Rebecca Sellers, who used to teach in the Shelby County Schools but decided to return to Memphis because she believed her skills were needed in the district's high-poverty schools, the experience has led her to consider leaving the profession. "All of a sudden this year I'm not doing it right," she said. "The joy of teaching is almost gone."

    Administrators in both districts say they plan to replace only a handful of teachers -- those at the very bottom of the ratings.

    "It's imperative for the district office to give schools and teachers the tools that they need," said Stephens, the Shelby assistant superintendent. "This evaluation thing is not a 'gotcha.' It's how can we help you improve."