Notes on a Remarkable Finding from Finland
I recently came across a paper by Ursina Schaede and Ville Mankki that contains a fascinating empirical finding with major implications for the way in which we think about meritocracy.
The paper examines the long run effects on students of a change in the manner in which their teachers were selected into a graduate program. Finland is well known for having an extremely effective school system, in part because primary teacher education has been “exclusively taught as a research-oriented, five year masters’ degree at universities” since the 1970s. These programs are in very high demand among applicants, with acceptance rates of about 10 percent. The admissions process has a first stage based largely on scores on a high school matriculation exam, followed by a second stage involving interviews and the evaluation of live teaching. Candidates are ranked again at the end of the process, and those at the top are taken until capacity is filled.
For a number of years, acceptance into the second stage was based on a quota, ensuring that at least 40 percent of students making it through the first stage of evaluation were male. Although this did not place any constraint on second stage outcomes, it turned out that the entering classes (and hence several cohorts of trained teachers) did not differ much in gender composition from those making it through the first stage. The quota was abolished in 1989, leaving first stage outcomes unconstrained. The first post-quota cohort thus graduated in 1994.
The paper examines the causal effects of this change on the long run outcomes for students. Identification is facilitated by variations across municipalities in the age distribution of teachers at the time of quota removal, coupled with mandatory retirement at age 60. This means that students were differentially exposed to the newer post-quota cohorts, which had a different gender composition (fewer males) and a different distribution of scores on the matriculation exam (higher scores on average).
The authors find that students differentially exposed to the quota-constrained cohorts of teachers ended up with better educational attainment and labor force participation at age 25. In other words, removal of the quota led to a decline in student performance. While this finding is interesting in its own right, even more interesting are the mechanisms that the authors rule out, and the one that they eventually accept.
Could the effect be arising through a role model channel, with particular benefit for boys? No, the authors find “no evidence for boys’ educational attainment being more affected from exposure to male quota teachers relative to girls,” and none of their “main effects differ systematically or significantly by pupil gender.”
Could it be that male and female teachers bring different benefits to the table, with the whole being greater than the sum of its parts? This would be a benefit from diversity. The authors cannot rule this out entirely (the estimates are too noisy) but they do find that “the benefits of adding an additional male teacher are similar in magnitude between places with few male teachers and places where the share of men among colleagues is already high.”
What, then, do the authors think is driving their results? They argue that the evidence “is consistent with male quota teachers contributing positive qualities to the school environment that are not sufficiently captured by the selection criterion in absence of the quota.”
It is important to be clear about this, because the finding can be so easily misunderstood. It is not that the quota teachers proved effective because they were male. It is that the distributions of important characteristics (unmeasured by scores) were not identical across male and female applicant pools. The quota was picking up individuals with these characteristics by proxy. It is the characteristics that mattered for students, not the gender of the teacher.
For example, male teachers in the data were “slightly more likely to come from rural areas and to live in their region and municipality of birth when compared to female teachers.” So a quota that favored rural applicants or those who had not moved from their municipality of birth could have had similar effects. In fact, this helps explain the mechanism—if rural applicants have fewer resources on average, they will have higher ability conditional on any given score than applicants from more resource-rich urban environments.
In fact, it is extremely likely that even within a given applicant group (men or women), the conditional distribution of these other valuable characteristics is not independent of score. There may be a particular range of scores at which these other characteristics happen to be especially abundant. In this case a policy that optimizes benefits for students may not even be monotonic within group—some people with higher scores would be skipped over in favor of those with somewhat lower scores.
This possibility is discussed at length in a recent paper with Rohini Somanathan that I will present at a symposium at Yale next week. The event has been organized by Gerald Jaynes and Rohini Pande, and is open to all (with registration).
Of course, policies that are non-monotonic (within group) would give rise to incentive effects, and probably could not be sustained. But the conceptual point they raise is that the understanding of meritocracy in public discourse is terribly impoverished. If one were to design a truly meritocratic policy, it could well have features that resemble the pursuit of representation targets. Meritocratic policies will not, in general, involve the application of a common score threshold across all candidates.
I understand that Ursina is on the academic job market this year and that this is the paper she’ll be presenting. I think that the work will be influential, and I wish her luck.
This might be too long a time for a response, but the paper was recently mentioned in the Economist and I found this blog post because I was curious if others had talked about it.
I think your premise about the quota being a proxy could certainly be correct, but the article just seems to give an implausible amount of weight to how students are accepted into the university as the only? mechanism for the success of a different cohert of students at least 30 years later. Schaede and Mankki are not talking a quota system for hiring teachers — they are talking about how Education majors get accepted into a university. They aren’t saying that student success at the university changed after the introduction of the quota. They are talking about the performance of 25 year olds whose primary school teacher was accepted into university under different systems (and it certainly complicates stats of matriculation exams when one takes into account that not all beginning Education majors will graduate with a degree in Education, and not all those that graduate will find a job, etc). I just can’t see how the data can be used to support the assertion, even if the underlying premise is correct. In effect, isn’t the article asserting that the education/employment achievements of a 25 year old is due to how their elementary school teacher got accepted to a university?
(As a factual issue, I doubt that retirement was mandatory at 60, but I arrived in Finland after the years in question so I don’t know that for sure. I suspect it was more that primary school teachers *could* retire at 60, with a reduction in pension if they hdn’t worked enough years).
Regarding "meritocracy," let me reiterate <a href="https://statmodeling.stat.columbia.edu/2007/12/04/meritocracy_won/">my point</a> (derived from James Flynn) that meritocracy won’t happen: the problem’s with the “ocracy.”