SIP:SIG Home

Officers

Members

Annual Meetings

Membership

Newsletter

Resources

AERA Home


 
 
 
 

School Indicators & Profiles SIG

A service to members of the American
Educational Research Association

The Importance of Identifying Levels in Multilevel Analysis: An Illustration of Ignoring Top or Intermediate
Levels in School Effectiveness Research


Marie-Christine Opdenakker
Jan Van Damme
K.U. Leuven


 

Introduction

In educational effectiveness research, multilevel models are increasingly used because these models take the multilevel structure of the data into account. In multilevel research, the data structure in the population is hierarchical (e.g. pupils in schools) and the sample data are viewed as a multistage sample from their hierarchical population. In such samples, the individual observations are generally not completely independent because of selection processes and because of the common history they share by belonging to the same group. Multilevel models take these dependencies into account (Hox, 1994).

The identification of a level above the individual level implies the existence of different levels of variation: if there are effects of the social context (the group level) on individuals, these effects must be mediated by intervening processes that depend on the characteristics of the social context. Groups can have a direct effect on individuals (different intercepts) or there can be cross-level interaction effects (different slopes). The latter requires the specification of processes within individuals that cause those individuals to be differentially influenced by certain aspects of the context.

However, what are the criteria for choosing levels and the number of levels? There are three kinds of criteria: the theory under investigation (Hox, 1994; Snijders & Bosker, 1996) or the research question, the kind of sampling used (cf. multistage sampling) (Hox, 1994; Snijders & Bosker, 1996) or the number of units belonging to a level e.g. when there are only three units in the highest level, it makes no sense to consider that level in the analysis. Sometimes, none of the criteria can be used to determine the (number of) levels. The international literature about multilevel modeling has paid very few or no attention to the determination of (the number of) levels nor to the effect of ignoring one or more levels in the analysis.
 

Purpose of the Study

The purpose of our contribution is to explore the effect of ignoring one or more levels of variation in hierarchical regression analysis. In a first analysis four hierarchical levels will be considered: the individual pupil, the class group, the teacher and the school. We will investigate the impact of ignoring the highest level (the school level), the highest two levels (the school and the teacher level) and ignoring one or two intermediate levels (the teacher and/or class level) on the attribution of the variance to the levels taken into consideration .
 

Data Sources and Method Used

The data set consists of longitudinal data of pupils and secondary schools in Flanders (Belgium), studied for a period of seven years (see Van Damme et al., 1996). We use a subsample of 2719 pupils in 154 classes, nested within 85 teachers in 49 schools. Mathematics achievement at the end of the common first grade is the dependent variable.

To analyze the data, the computer program MLn (Woodhouse, 1995) was used. By manipulating the number of levels and the kind of level ignored (top level or intermediate level) and comparing the results of these models with the four level model, we investigated the implications of ignoring levels of variation. The results of the four level model indicates that each level is important: 54.39% of the overall variance in mathematics is linked to the individual level, 14.68% to the class level, 17.38% to the teacher level and 13.55% to the school level. Exemplary, also models with a relevant independent variable at each level will be used to show the effect of ignoring levels on the effect size of main effect variables.
 

References

Hox, J.J. (1994). Applied multilevel analysis: TT-publicaties.

Snijders, T.A.B., & Bosker, R.J. (1996). Introduction to multilevel analysis. London: Sage. (in press).

Van Damme, J., De Troy, A., Meyer, J., Minnaert, A., Lorent, G., Opdenakker, M.-C., & Verduyckt, P. (1996). De aanvangsjaren in het secundair onderwijs. Een eerste bundeling van resultaten van het LOSO-project. Leuven: K.U.Leuven, Onderzoekscentrum voor Secundair en Hoger Onderwijs.

Woodhouse, G. (Ed.) (1995). A guide to MLn for new users. London: University of London, Institute of Education.
 
 

Marie-Christine Opdenakker
Vesaliusstraat 2
B-3000 Leuven
Belgium


Hosted by:
Learning Point Associates