Measuring School Leaders' Effectiveness: An Interim Report from a Multiyear Pilot of Pennsylvania's Framework for Leadership

Publisher: Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, REL Mid-Atlantic
Dec 17, 2014
Authors
Bing-ru Teh, Hanley Chiang, Stephen Lipscomb, and Brian Gill

Key Findings:

  • School leaders who earned higher scores in one category of leadership practices measured by the Framework for Leadership tended to earn higher scores in the other categories.
  • On each measured leadership practice most school leaders earned scores in the top two of four possible performance levels.
  • School leaders with larger estimated contributions to student achievement growth did not score higher than school leaders with smaller estimated contributions.

This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania’s tool for evaluating the leadership practices of principals and assistant principals. The study analyzed three key properties of the FFL: internal consistency, score variation, and concurrent validity. To measure the internal consistency of the FFL, Cronbach’s alpha was calculated for the full FFL and for each of its four categories of leadership practices. Score variation was characterized by the percentages of school leaders earning scores in different portions of the rating scale. Concurrent validity was assessed through a regression model for the relationship between school leaders’ estimated contributions to student achievement growth and their FFL scores. Based on a pilot in which 336 principals and 69 assistant principals were rated by their supervisors in 2012/13, this interim report finds that the full FFL had good internal consistency for both principals and assistant principals. However, most scores for specific leadership practices were in the top two of four possible performance levels, and FFL scores were not associated with school leaders’ contributions to student achievement growth. These findings suggest that more evidence is needed on the validity of using FFL scores to identify effective and ineffective school leaders.