I have no strong opinion on what would be the best admissions strategy for schools like Dartmouth, but I think your discussion elides one relevant statistical consideration. In particular, test scores are noisy signals of underlying "ability" or "testiness". In such circumstances, the average test score of an individual's group is relevant to estimating their true "testiness" conditional on their test score. You - and many admissions officers - conclude that, among two students that both score 1400 on a max-1600 test, the kid from a disadvantaged background has more latent ability than the one from an advantaged one. Maybe, if you feel that the talent is undeveloped, but from the standpoint of the noisy test, you'd have to conclude the opposite, i.e., the kid from the disadvantaged background is more likely to have had a lucky score and that their "true score" is lower than that of the wealthier kid.
Your point is valid, and I'm aware of it. I recall a paper by JP Benoit making a similar point but can't remember the exact reference. What this means is that there are countervailing forces and the net effect is theoretically ambiguous. If you look at the report you will see that first year GPA is lower for students from disadvantaged backgrounds even conditional on score, so if they have higher latent ability it reveals itself later than this, possibly even after graduation and into careers. I do believe that this is the case but aside from some anecdotal evidence discussed in my last post I don't know if any empirical test. It's also possible that success depends more on network effects than latent ability, this is discussed in my recent JEL paper. In this case legacies will over perform conditional on score.
If one took the social planner's problem to be to maximize some combination of GDP and intergenerational mobility (which is kind of what I would hope), then what would the optimal admissions policy? It would seem to depend in part on the two issues discussed above - i.e., to what extent does group identity signal latent talent versus noisiness in test scores. What else would it depend on? The extent to which admissions to elite schools confers mobility in and of itself? The extent to which the right matching of educational opportunities influences societal productivity? My own biases are to think that the answers to these last two questions are that elites schools don't matter all that much for subsequent success and that the choices over who goes there do not tend to affect aggregate productivity.....but there is obviously room for debate. Have you written on anything resembling such a social planner's problem and, if so, what other desiderata did you consider?
I think the the idea that requiring tests scores is paternalistic is a little ridiculous. A college sets all sorts of requirements that students must meet for the application process and judges applications on criteria that it alone sets.
Transparency about how it weighs scores based on student background through a few examples can help get more applications if the concern is that fewer students from lower class backgrounds will apply if they are required to submit scores.
Yes I got that. But I see that my reply didn't reflect that. I simple mean that I disagree with the framing that the Dartmouth is protecting them from their own mistakes. They are simply instituting requirements that they feel will get them the best data about applicants. To view this as a commentary on the judgment of certain applicants is unnecessary and unjustified in my opinion.
Transparency about the admissions process, preferably with anonymized data showing the weight they put on on neighbourhood and school characteristics should serve to explain their policies well enough. The sort of students vying for hihgly sought after spots are sophisticated enough to understand this sort of system.
I have no strong opinion on what would be the best admissions strategy for schools like Dartmouth, but I think your discussion elides one relevant statistical consideration. In particular, test scores are noisy signals of underlying "ability" or "testiness". In such circumstances, the average test score of an individual's group is relevant to estimating their true "testiness" conditional on their test score. You - and many admissions officers - conclude that, among two students that both score 1400 on a max-1600 test, the kid from a disadvantaged background has more latent ability than the one from an advantaged one. Maybe, if you feel that the talent is undeveloped, but from the standpoint of the noisy test, you'd have to conclude the opposite, i.e., the kid from the disadvantaged background is more likely to have had a lucky score and that their "true score" is lower than that of the wealthier kid.
Your point is valid, and I'm aware of it. I recall a paper by JP Benoit making a similar point but can't remember the exact reference. What this means is that there are countervailing forces and the net effect is theoretically ambiguous. If you look at the report you will see that first year GPA is lower for students from disadvantaged backgrounds even conditional on score, so if they have higher latent ability it reveals itself later than this, possibly even after graduation and into careers. I do believe that this is the case but aside from some anecdotal evidence discussed in my last post I don't know if any empirical test. It's also possible that success depends more on network effects than latent ability, this is discussed in my recent JEL paper. In this case legacies will over perform conditional on score.
Thank you for the comment.
If one took the social planner's problem to be to maximize some combination of GDP and intergenerational mobility (which is kind of what I would hope), then what would the optimal admissions policy? It would seem to depend in part on the two issues discussed above - i.e., to what extent does group identity signal latent talent versus noisiness in test scores. What else would it depend on? The extent to which admissions to elite schools confers mobility in and of itself? The extent to which the right matching of educational opportunities influences societal productivity? My own biases are to think that the answers to these last two questions are that elites schools don't matter all that much for subsequent success and that the choices over who goes there do not tend to affect aggregate productivity.....but there is obviously room for debate. Have you written on anything resembling such a social planner's problem and, if so, what other desiderata did you consider?
I don't think I have anything specifically on these questions but they are interesting and I'll give it some thought.
Thanks for your work on this topic. One of my first substack posts was on Frank Bruni's college admissions book (https://subgameperfect.substack.com/p/et-tu-bruni)...perhaps of interest and an easy, quick read.
Link broken, post again?
https://subgameperfect.substack.com/p/et-tu-bruni
This is extremely interesting and is in line with reporting from the NY times in this podcast episode: https://www.nytimes.com/2024/01/17/podcasts/the-daily/the-war-on-the-sat.html
I think the the idea that requiring tests scores is paternalistic is a little ridiculous. A college sets all sorts of requirements that students must meet for the application process and judges applications on criteria that it alone sets.
Transparency about how it weighs scores based on student background through a few examples can help get more applications if the concern is that fewer students from lower class backgrounds will apply if they are required to submit scores.
Requiring tests is not paternalistic. Requiring them to protect applicants from their own mistakes is paternalistic.
Yes I got that. But I see that my reply didn't reflect that. I simple mean that I disagree with the framing that the Dartmouth is protecting them from their own mistakes. They are simply instituting requirements that they feel will get them the best data about applicants. To view this as a commentary on the judgment of certain applicants is unnecessary and unjustified in my opinion.
Transparency about the admissions process, preferably with anonymized data showing the weight they put on on neighbourhood and school characteristics should serve to explain their policies well enough. The sort of students vying for hihgly sought after spots are sophisticated enough to understand this sort of system.
Whether or not they are sophisticated enough is an empirical question and my proposed solution will generate the data necessary to answer it.