Search

Areas of Expertise

 

Achievement Gaps
Literature Library

Topics

Why Do the Achievement Gaps Exist?

  1. Explanations

    1. School Factors

      1. Teacher Expectations and Practices

Teachers Make a Difference in Student Performance

A researcher argues that much of the research on the influence of schools and teachers on student achievement is flawed. Because of a lack of good data and because of methodological problems, previous research has downplayed the importance of school and teacher influences in student achievement. However, by using newer statistical methods and data, the author finds that schools and teachers can make a significant difference in student achievement.

Citation:
This reports some of the ideas and findings from the following source:

Wenglinsky, H. (2001).Teacher classroom practices and student performance: How schools can make a difference.Retrieved August 28, 2002, from Educational Testing Service Web site:http://www.ets.org/research/researcher/RR-01-19.html.
Get Adobe Reader

Adobe Reader FAQ

To see other reports that originated from this same citation, go to the bibliography.

On its face, it seems obvious. Teachers make a difference when it comes to student achievement.

A puzzling fact, however, is that the majority of quantitative research seems to indicate that it is not so much teachers or schools that make a difference in student achievement, but it is student characteristics and things outside school (like family characteristics) that really matter.

Researcher Harold Wenglinsky thinks that previous quantitative research has missed the importance of teachers and misconstrued some very important factors for student achievement. Why? Wenglinsky argues that previous research on academic achievement has suffered from a lack of good data and some serious methodological problems.

Problems with Research on Teacher Effectiveness

The issue, thinks Wenglinsky, is not that teachers are not important. It is just that there have not been good ways to measure teacher importance when using large data sets and statistical techniques for modeling what is going on. Wenglinsky identifies four problems with previous research that cause the studies to seriously underestimate teacher influence.

  1. Lack of good data. Large data sets rarely have detailed information on classroom practices (for example, whether hands-on learning activities are used, whether higher order thinking skills are taught, and so on). Many previous studies rely on "teacher input" variables (for instance, the teacher's level of education), without being able to study what the teachers actually did in the classroom.
  2. Insensitivity to multilevel aspects of education. Commonly used statistical techniques are not sensitive to the multilevel nature of student achievement. For instance, affluent kids may perform better than less-affluent kids at the same school, but these effects are different from when students at mostly affluent schools outperform students at less-affluent schools. In the second case, there are influences that the school has quite apart from whether the particular student is rich or poor. Studies that don't distinguish between the different levels of influence "can miss important information about the nature of school effects," says Wenglinsky.
  3. Failure to take measurement error into account. Previous statistical models fail to take measurement error into account, Wenglinsky indicates. This means that coefficients of different variables will be biased. In other words, when measurement error is not taken into account, the statistical models will be mistaken about what factors are really important.
  4. No good measurement of interrelationship of independent variables. Previous statistical techniques are not very good at studying the relationships between independent variables. In other words, several causes go into explaining academic achievement. However, previous studies could not easily see how different causes influenced each other.

A More Sensitive Approach to Modeling

Wenglinsky says that new data sets and newer methods of statistical analysis now allow us to get around these limitations.

  • Recent test data from the National Assessment of Educational Progress gather information not only on student achievement but on a range of classroom practices that teachers use as well.
  • A technique called multilevel structural equation modeling allows researchers to study how different levels (for example, student and school factors) influence student outcomes and each other.
  • Wenglinsky is careful about taking measurement error into account.
  • Structural modeling techniques allow researchers not only to tell whether a particular factor influences the student outcome but also how the factor interacts with other factors.

What Will These Alternative Methods Show Us?

Because this multilevel structural equation modeling allows us to get around some of the problems of previous research, Wenglinsky believes that we will now be able to get a better sense of a teacher's influence. Before, the strength of a teacher's influence on a child's academic achievement was "hidden" because of the methodology.

What does Wenglinsky expect to see?

First, Wenglinsky breaks the concept of "teacher quality" down into three parts:

  1. classroom practices
  2. professional development
  3. a teacher's characteristics external to the classroom (for instance, level of education)

Wenglinsky hypothesizes that of all these, classroom practices will be the most important, since this is where teachers and students interact.

Second, Wenglinsky expects that the effects of teacher quality on student performance will be as strong as student background characteristics.

What Does Wenglinsky's Research Find?

Wenglinskly runs a separate school-level model for each part (or factor) of teacher quality: teacher inputs, professional development, and classroom practice. This allows him to determine not only which variables make a contribution to each factor, but it also allows him to determine the relative importance of each factor for influencing student achievement.

This way, Wenglinsky can tell not only how important a teacher's classroom practice is relative to his or her level of education, but also how important teacher influence in general is relative to student background characteristics.

Model

What factors were important?

What is the effect size?

Teacher Input Model

A teacher's major (in this case, whether a math teacher was also a math major in college) was weakly associated with student achievement.

0.09

Professional Development Model

Two factors were substantially related to student achievement:

  • professional development that addresses special student populations
  • professional development that focuses on teaching higher order thinking skills

0.33

Classroom Practice Model

Three constructs were positively related to student achievement:

  • teachers using hands-on learning practices
  • teachers using higher order problem-solving approaches
  • teachers using authentic assessments (such as portfolios as opposed to traditional tests) of students' work

0.56

Total Effect Size: 0.98

What do these results say in terms of Wenglinsky's two hypotheses?

Hypothesis 1: Teacher classroom practices will be the most important aspects of a teacher's influence on student achievement. Wenglinsky concludes that with an effect size of 0.56, the classroom practices do in fact have the largest effect of all teacher factors. Hypothesis 2: Teacher influence will have a larger influence on student achievement than student SES. Wenglinsky also compares the total effect size of the school levels models (0.98) to the total effect size of the student socioeconomic status (SES) models (0.76). This allows him to conclude that the impact of teaching is not only comparable to SES, but even somewhat greater.

Interactions Among Independent Variables

However, Wenglinsky points out that understanding just how teaching is related to student achievement requires that we understand how the different factors relate to each other. By using multilevel structural equation modeling, Wenglinsky finds that many of the causal variables are associated with each other.

  • Schools with a high percentage of affluent students tend to spend less time on teacher professional development generally.
  • Schools with a high percentage of affluent students are also less likely to expose their teachers to professional development concerned with working with different student populations (that is, professional development in student diversity).
  • Schools with more affluent students are more likely to have teachers who use higher order teaching methods.
  • Schools with more affluent students are less likely to have teachers who engage in inauthentic (that is, traditional testing methods) forms of assessment.
  • Schools where teachers have received professional development in working with different student populations are less likely to have their students engaged in routine problem solving (lower order teaching methods).
  • Schools where teachers have received professional development in higher order thinking skills are more likely to have students engage in hands-on learning.
  • The more time that teachers engage in professional development, the more likely their students are to engage in hands-on learning and authentic assessment.

This web of interrelationships among independent variables is mapped in Figure 1.

Figure 1. Interrelationships between School and Teaching Factors that Influence Student Achievement

The Bottom Line

Wenglinsky concludes that when class size and student affluence are taken into account,five aspects of teacher quality tended to promote student achievement:

  1. teacher's major same as subject being taught
  2. professional development in higher order thinking skills
  3. professional development in student diversity
  4. using hands-on learning methods in the classroom
  5. encouraging higher order thinking skills in the classroom

Wenglinsky says that this tells us that "passive" teachers (who reduce teaching to its simplest components) basically leave students to perform as well as the students' backgrounds will allow. "Active" teachers, in contrast, provide a real added value by pressing all students to grow regardless of their background.

He says that in schools that lack a critical mass of "active" teachers, teacher and school characteristics probably do not matter much for student achievement. However, in schools that do have a critical mass of active teachers, schools and teachers add significant value to student academic performance.

Research Design:

Research Questions

What kinds of influences do schools and classroom practices have on student achievement?

What kinds of classroom practices seem to have the most influence on student achievement?

Data

Wenglinsky draws on NAEP 1996 8th grade math score data. The NAEP data, in addition to including an assessment of math skills also includes background questionnaires completed by the student, the teacher in the relevant subject area, and the principal. The teacher questionnaire includes information on classroom practices.

Method

Wenglinsky uses a multilevel structural equation modeling technique. Using this technique, he can distinguish between student-level factors and school-level factors.

Structural modeling involves two components:

  1. a factor model: this model allows Wenglinsky to determine which variables contribute to a given factor (a sort of composite independent variable constructed from a number of more measurable—"manifest"—variables).
  2. a path model: this model is able to relate all the different factors to each other.

Even though the multilevel structural equation modeling allows Wenglinsky to overcome a number of methodological limitations of earlier research on school effects, he notes four limitations to his own method.

  1. Because the data is cross-sectional, it is impossible to be sure of the direction of causation.
  2. The study covers only one subject in one grade. It is entirely possible that classroom influences may matter differently for different subjects as well as different grades of students.
  3. The method does not study the link between aspects of teacher quality and the relationship between student SES and student test scores. So, it is not possible to measure the relationship between a school variable and the relationship between two student-level characteristics.
  4. While the measures of different aspects of teacher quality are better than much information we have before now, Wenglinsky notes that we still need better ways to measure the different factors or constructs of teacher quality.
Funding

This research was funded by the Milken Family Foundation.

 



Copyright © 2007 Learning Point Associates. All rights reserved.