Look I’m from New Zealand - so when everyone around me started talking about ATARs I just smiled and nodded. In fact I probably couldn’t talk to most people in New Zealand about education. I’m from the “pre-NCEA” era - where a single end of year exam for five courses, scaled to fit a within-course normal distribution, determined most of anything. As a result, these more complex design criteria are well outside of my lived experience.
Hey Matt, great read. I wonder what you intuitively think about the distinction between SAT and ATAR and how they might influence inequality in university admissions. I’m a pretty big fan of the ATAR as a system since it controls for subject difficulty fairly robustly but also incorporates a variety of in-class assessments (so you dodge the argument of having someone’s life hinge on a single assessment - though it seems like a fairly weak argument based on the results of the note).
With that said, I’d be interested in how incorporating school-level assessments in the ATAR affects its robustness to inequality, relative to the SAT which doesn’t include any in-school measurement. Intuitively it seems like it might “double count” for the selection problem where high-SES students go to better schools. At the same time, the ATAR weights the in-school portion based on the standardised portion so maybe the effect is accounted for. In any case I’d be keen to hear views on how the in-school part of the ATAR stacks up w.r.t inequality.
That's a good point - as I noted I know nothing about education or the extensive evidence on these issues, so can only really speculate. And when speculating all I can really chat about is hypothetical trade-offs.
The inclusion of significant internal assessment components in NZ was based on the idea that i) they were a better test of knowledge ii) single exam assessment tended to favour students with the resources to prepare for the exams.
However, as you note there is an issue with this view - high means students are also the ones that will be able to get advice and guidance on internal assessment. In fact, practicing for a standardised test may be more open to preparation across the income spectrum than support for each individual piece of assessment.
Also I'm quite glad I was a student when externally assessed exams could determine your marks. At my high school there were many things going on, and one of my courses (not all) the teacher absolutely hammered me on internal assessment - but I ended up with a good mark after acing the test. Although I wasn't a perfect student, there was definitely non-objective reasons for some of this assessment - and external assessment removes this type of discrimination in grades (feels closer to an objective measure of understanding material).
As I have no experience with ATAR I'm not sure how all these things function. However, I do realise there is no such thing as a perfect assessment criteria for everyone - and ultimately these criteria are designed with these trade-offs in mind, and should be viewed as an imperfect proxy of a students type.
I think there is a tendancy among absolutely everyone - especially people who work in service/intellectual disciplines - to picture a counterfactual where the assessment criteria slightly improved their ability to access opportunities, and then picture what they would have done with that. However, these counterfactuals are hard, and lifepaths are filled with uncertainty - for many social inequities it may not be changes in assessment criteria that are the best policy solution, but instead simply giving people access to resources and a safety net when they take risks.
Hey Matt, great read. I wonder what you intuitively think about the distinction between SAT and ATAR and how they might influence inequality in university admissions. I’m a pretty big fan of the ATAR as a system since it controls for subject difficulty fairly robustly but also incorporates a variety of in-class assessments (so you dodge the argument of having someone’s life hinge on a single assessment - though it seems like a fairly weak argument based on the results of the note).
With that said, I’d be interested in how incorporating school-level assessments in the ATAR affects its robustness to inequality, relative to the SAT which doesn’t include any in-school measurement. Intuitively it seems like it might “double count” for the selection problem where high-SES students go to better schools. At the same time, the ATAR weights the in-school portion based on the standardised portion so maybe the effect is accounted for. In any case I’d be keen to hear views on how the in-school part of the ATAR stacks up w.r.t inequality.
That's a good point - as I noted I know nothing about education or the extensive evidence on these issues, so can only really speculate. And when speculating all I can really chat about is hypothetical trade-offs.
The inclusion of significant internal assessment components in NZ was based on the idea that i) they were a better test of knowledge ii) single exam assessment tended to favour students with the resources to prepare for the exams.
However, as you note there is an issue with this view - high means students are also the ones that will be able to get advice and guidance on internal assessment. In fact, practicing for a standardised test may be more open to preparation across the income spectrum than support for each individual piece of assessment.
Also I'm quite glad I was a student when externally assessed exams could determine your marks. At my high school there were many things going on, and one of my courses (not all) the teacher absolutely hammered me on internal assessment - but I ended up with a good mark after acing the test. Although I wasn't a perfect student, there was definitely non-objective reasons for some of this assessment - and external assessment removes this type of discrimination in grades (feels closer to an objective measure of understanding material).
As I have no experience with ATAR I'm not sure how all these things function. However, I do realise there is no such thing as a perfect assessment criteria for everyone - and ultimately these criteria are designed with these trade-offs in mind, and should be viewed as an imperfect proxy of a students type.
I think there is a tendancy among absolutely everyone - especially people who work in service/intellectual disciplines - to picture a counterfactual where the assessment criteria slightly improved their ability to access opportunities, and then picture what they would have done with that. However, these counterfactuals are hard, and lifepaths are filled with uncertainty - for many social inequities it may not be changes in assessment criteria that are the best policy solution, but instead simply giving people access to resources and a safety net when they take risks.