Across the mathematics education research community, it is now generally accepted that teachers should support students to engage with rich mathematical tasks, develop meaningful understandings of and connections between mathematical concepts, procedures, and representations, and collaboratively construct and communicate their mathematical thinking (NCTM 2000, 2006).  In support of these goals, research has identified components of high-quality, or ambitious, mathematics instruction that are likely to support students’ learning of mathematics with understanding (Franke, Kazemi, & Battey 2007; Hiebert et al. 1993; Stein, Smith, Henningsen, & Silver 2000; USDE 2000).  However, relationships between ambitious mathematics instruction and student achievement in mathematics remain relatively unexamined in the literature.

In this study, we use the Instructional Quality Assessment (IQA) Mathematics Classroom Observation Toolkit as an observational assessment system to examine relationships between several meaningful aspects of ambitious mathematics instruction and students’ performance on state mathematics achievement tests.  The IQA is a measurement tool designed to identify pathways for instructional improvement (Boston, in press; Matsumura et al. 2008).  The IQA (as used in this study) contains classroom observations rubrics for assessing the presence and quality of ambitious instruction based on the constructs of Academic Rigor and Accountable Talk (Resnick & Hall 2001). Academic rigor assesses students’ opportunities to engage in rigorous thinking about challenging content. Accountable talk upholds the standards of the discipline for accuracy, evidence for claims, or reasoning behind ideas and conjectures, while also responding to, developing, and advancing the knowledge, ideas, and claims of all students in the classroom.  This paper explores the following research question: “How are gains in student achievement related to teachers’ ability to enact the dimensions of high-quality mathematics instruction assessed by the IQA?”

The data analyzed in this paper comes from a four-year study that was designed to help understand what it takes to improve the quality of middle-grades mathematics teaching, and thus student achievement, at the scale of a large urban district. The four participating districts face the typical challenges of large urban districts in the U.S.: serving large numbers of historically low-performing and underserved groups of students in mathematics, having limited resources and under-prepared teachers, and high rates of teacher turnover (Darling-Hammond, 2000).  These districts were selected in part because they intend to improve student achievement in middle grades mathematics by supporting teachers’ development of ambitious instructional practices, and each district has devoted significant resources to implementing a variety of improvement strategies.

In each district, we selected a representative sample of 6 to 10 middle grades schools, and within each school, we randomly selected up to five mathematics teachers to participate in the study, for a total of approximately 30 teachers per district.  For each participating teacher, we documented instructional practices by videotaping two consecutive classroom sessions with the same group of students, for a total of approximately 240 classroom sessions per year.  Hence, we present findings based on the analysis of video-recordings of approximately 750 classroom sessions taught by 204 different teachers across 30 schools over the first three years of the study.  Each district provided individual student achievement data linked to teachers’ classrooms, including student demographics and each student’s current and prior two years of scores on annual state mathematics and reading assessments.

Video-recordings of classroom sessions were coded using the IQA Lesson Observation rubrics.  A value-added model (VAM) was developed to produce a score for each participating teacher in each year of the study representing how much higher or lower a hypothetical “average student” that year would be expected to score on an achievement test if the student were taught by that teacher, as compared with all other teachers participating in the study.  The VAM controlled for each student’s prior two years of mathematics achievement scores and student- and class-level covariates determined in the model and in the literature as possibly impacting a teacher’s capacity to provide high-quality instruction.

In this study, we examine correlations between teachers’ scores on individual dimensions of the IQA measure and the VAM estimates.  Results indicate that instructional practices in the observed classrooms typically receive low scores for many dimensions of the IQA.  The cognitive demand of instructional tasks used in teachers’ classrooms is generally high, though the level of rigorous thinking in which students actually engage while working on tasks is considerably lower and many teachers provide limited opportunities for students to engage in high-quality whole-class discussions.  In general, we see little change in the IQA measures across the three years of the study.  When we examine the strength of the relationships between the ranks of teachers’ value-added estimates and the ranks of the various IQA measures, we generally see small positive correlations.  However, we also see considerable heterogeneity among the relationships depending on the district and year, warranting further examination of the relations between the IQA measures and the VAM estimates.  In this paper, we explore hypotheses for the differences in correlations across districts and years.

While previous studies have examined relationships between ambitious mathematics instructional practices with student achievement on state or national assessments, many of these studies are based on teachers’ self-reports on questionnaires or surveys (e.g., Hamilton & Martinez 2007; Le et al. 2009; McCaffrey et al. 2001).  Other studies have utilized measures of ambitious instruction involving direct or videotaped classroom observations (e.g., Schoen et al., 2003; Saxe, Gearhart, & Seltzer; 1999; Tarr et al., 2008; Weiss, Pasley, Smith, Banilower, & Heck, 2003). Previous studies also differ in what they consider to be indicators of ambitious instruction and whether they evaluate the quality (vs. the presence) of these indicators.

Schools and districts across the U.S. are under pressure to increase test scores, and without evidence indicating that improving the quality of mathematics instruction will result in higher test scores, they often respond to accountability pressures by teaching to the test or gaming the system.  By combining the use of the IQA as an assessment system with student achievement data, researchers can guidance for districts, schools, and teachers who seek to improve learning opportunities for students.