Help - Search - Members - Calendar
Full Version: Statistical pass rate
Forums > ABRSM > Diplomas
RoseRodent
Someone posted the link to exam stats on another thread and I happened to look at the diploma stats too. Wish I hadn't!
http://www.abrsm.org/en/press/factfile/diplomastats/

Does the board have a requirement for passing only around 50% of candidates regardless of the standard presented, or are there really that many candidates showing up who are so short of the mark? I find it hard to believe that somehow only 9% of candidates fail at grade 8 and then consistently year on year 50% or thereabouts fail their diplomas. I know there is a big gap between grades and diplomas, but seriously, fail rates are between 41 and 49%!! What's going on there? Poor preparation? Not getting access to experienced teachers at this level? Not submitting the correct material? The mind boggles how nearly half of candidates can be failing consistently year on year. Anyone know more about it? I'm really hoping to get to the AB conference this year where there is a session on diplomas, I will definitely bring that up, but does anyone know?

Susie
I think the QS is rather under-prepared in some (perhaps a lot! of) cases.
owainsutton
Having neither taken a diploma nor prepared a pupil for it, I can't speak from experience, BUT.....how on earth could they impose such a limit? Give each examiner a quota? "You can only pass two of today's candidates, the rest have to fail, because we're over budget"?
flobiano
QUOTE(RoseRodent @ Jun 27 2012, 11:41 AM) *

Someone posted the link to exam stats on another thread and I happened to look at the diploma stats too. Wish I hadn't!
http://www.abrsm.org/en/press/factfile/diplomastats/

Does the board have a requirement for passing only around 50% of candidates regardless of the standard presented, or are there really that many candidates showing up who are so short of the mark? I find it hard to believe that somehow only 9% of candidates fail at grade 8 and then consistently year on year 50% or thereabouts fail their diplomas. I know there is a big gap between grades and diplomas, but seriously, fail rates are between 41 and 49%!! What's going on there? Poor preparation? Not getting access to experienced teachers at this level? Not submitting the correct material? The mind boggles how nearly half of candidates can be failing consistently year on year. Anyone know more about it? I'm really hoping to get to the AB conference this year where there is a session on diplomas, I will definitely bring that up, but does anyone know?


I've always assumed that it was because it was a high standard to achieve and many people underestimate the level required. You also have to pass all 3 sections to pass overall, I get the impression that it is quite hard to pass all three at one sitting and many people have to redo just one section. Not sure how this is included in the stats but could skew them slightly if some people decide not to resit if they have passed the recital bit.

Complete conjecture but I would think most instrumental teachers have good experience of Grade 8 and what is required, but they may be likely to have fewer diploma candidates or maybe candidates are more likely to tackle these exams without a regular teacher and underestimate the requirements?

I also think the stats also cover all 3 levels of diploma so it could be that failure rates are higher for higher dips too - I get the impresssion that the Fellowship is VERY difficult to pass.
Deborah
Remember as well that each section has to be passed - it's not just the final score. I suspect that the pass rate for grade exams would be lower if each section had to be passed, and the pass rate for diplomas would be higher if one reached the total pass mark by any fashion. I would have passed my Dip first time round if the result was based on just the total mark, but I failed the Viva, so first time round counted as a fail.
flobiano
Another random thought probably linked to Deborah's observation.

Although only 9% of people fail Grade 8, only about 50-60% get merit or distinction which is roughly the same percentage of people who pass the diploma. So I think the pass mark on a diploma could also be thought of as a raising of the bar so that you need to be at least "merit" level to pass.

Hopefully that makes some sort of sense. unsure.gif
floboe
I personally don't think people really know the difference between the the diplomas and grades when they apply. In the diploma you can't just get by by playing the pieces, you have to add your own interpretation and have much stronger technique amongst other things.

I looked at these statistics before hand, which didnt help, after the exam I was so worried but thought that if only half of the people that enter pass, then it isn't so much of a problem if I didnt pass all the stages that time. ( thankfully I did pass everything first time, it would have been bad if I had to re-enter for parts and my twin sister had passed wacko.gif )

The Viva Voce looks worse than it really is but you do need to know a lot about your instrument and the pieces of music, it means a lot of revision and research.

I don't think there is any requirement for passing only 50% but as you say, there are so many people who fall short of the 40% pass mark, you have to pass each section to pass the whole thing, so if you get less than 6 on the sightreading but you have a lot more than the required pass on the performance section, you have to retake that one part to get the letters.

Hope this all makes sense!
RoseRodent
QUOTE(owainsutton @ Jun 27 2012, 11:51 AM) *
how on earth could they impose such a limit? Give each examiner a quota? "You can only pass two of today's candidates, the rest have to fail, because we're over budget"?


There are some examinations that are done in just that manner, or have been in the past. It used to be that with certain national qualifications (GCSE when it first came out, as I recall, may have been only certain boards) that there was a set level for passing, beyond that they would compare candidates as a cohort and the top 5% of all marks got an A, the next % a B... So if one year you took an A level and got 94% but lots of other people got 99% you would have a B. The following year you could get 89% but if nobody else got higher than 90% that would be an A. This was supposed to even out differences in the difficulty of the paper year on year. It's certainly something that is practiced, whether we recognise it or not, in every field of interview and audition - if you are recruiting for 2 people then you could get 10 turn up of whom 4 are excellent but you can only take 2, or you could get 10 turn up of whom you have to take the 2 best to fill the position but really wish you didn't have to! It's a more common method of administration than we realise, so it's absolutely possible that marks would be adjusted during a moderation process.

I might ask the office as to how the first time fail second time partial resit pass is recorded, whether it's recorded as a pass in the following session or not. It's possible they aren't fully reflected in the stats at all.
vee
QUOTE(Deborah @ Jun 27 2012, 10:56 AM) *

Remember as well that each section has to be passed - it's not just the final score. I suspect that the pass rate for grade exams would be lower if each section had to be passed, and the pass rate for diplomas would be higher if one reached the total pass mark by any fashion. I would have passed my Dip first time round if the result was based on just the total mark, but I failed the Viva, so first time round counted as a fail.


How does TG compare, any idea? Since you dont have viva or QS, once would think that its easier- somehow I don't think thats quite true. Since you're marked only on your performance, I suppose you have to be completely confident about your programme since there is no other way to compensate for loss of marks in this section
barry-clari
QUOTE(vee @ Jun 27 2012, 12:24 PM) *

QUOTE(Deborah @ Jun 27 2012, 10:56 AM) *

Remember as well that each section has to be passed - it's not just the final score. I suspect that the pass rate for grade exams would be lower if each section had to be passed, and the pass rate for diplomas would be higher if one reached the total pass mark by any fashion. I would have passed my Dip first time round if the result was based on just the total mark, but I failed the Viva, so first time round counted as a fail.


How does TG compare, any idea? Since you dont have viva or QS, once would think that its easier- somehow I don't think thats quite true. Since you're marked only on your performance, I suppose you have to be completely confident about your programme since there is no other way to compensate for loss of marks in this section


I don't think TG publish their results publicly, vee : my suspicion though is that the pass/fail ratio is similar to ABRSM, not withstanding TG diplomas only having two sections, and you don't have to pass both sections to pass overall.

Bear in mind : all diplomas are difficult! It is only to be expected that a significant proportion of musicians that attempt them won't pass!
owainsutton
QUOTE(RoseRodent @ Jun 27 2012, 12:08 PM) *

QUOTE(owainsutton @ Jun 27 2012, 11:51 AM) *
how on earth could they impose such a limit? Give each examiner a quota? "You can only pass two of today's candidates, the rest have to fail, because we're over budget"?


There are some examinations that are done in just that manner, or have been in the past. It used to be that with certain national qualifications (GCSE when it first came out, as I recall, may have been only certain boards) that there was a set level for passing, beyond that they would compare candidates as a cohort and the top 5% of all marks got an A, the next % a B... So if one year you took an A level and got 94% but lots of other people got 99% you would have a B. The following year you could get 89% but if nobody else got higher than 90% that would be an A. ... it's absolutely possible that marks would be adjusted during a moderation process.

Shifting grade boundaries for exams is one thing, but adjusting marks after the fact is completely different. What criteria would decide which candidates' marks get changed, and by how much?
vee
QUOTE(barry-clari @ Jun 28 2012, 07:48 AM) *

QUOTE(vee @ Jun 27 2012, 12:24 PM) *

QUOTE(Deborah @ Jun 27 2012, 10:56 AM) *

Remember as well that each section has to be passed - it's not just the final score. I suspect that the pass rate for grade exams would be lower if each section had to be passed, and the pass rate for diplomas would be higher if one reached the total pass mark by any fashion. I would have passed my Dip first time round if the result was based on just the total mark, but I failed the Viva, so first time round counted as a fail.


How does TG compare, any idea? Since you dont have viva or QS, once would think that its easier- somehow I don't think thats quite true. Since you're marked only on your performance, I suppose you have to be completely confident about your programme since there is no other way to compensate for loss of marks in this section


I don't think TG publish their results publicly, vee : my suspicion though is that the pass/fail ratio is similar to ABRSM, not withstanding TG diplomas only having two sections, and you don't have to pass both sections to pass overall.

Bear in mind : all diplomas are difficult! It is only to be expected that a significant proportion of musicians that attempt them won't pass!


Oops Barry sad.gif Wrong highlights for me to read right now- with my own exam next week !

I've also wondered how you can pass Viva alone as a separate section. Doesn't it have to do with your programme and how well versed you are with what you have written in your programme notes and its context? So if you pass viva first time around - but fail the performance section, next time you may be putting together a different programme. Somehow it makes more sense to link both the recital and viva together - but QS of course is an independent test- no question about that.
barry-clari
QUOTE(vee @ Jun 28 2012, 11:50 AM) *


Oops Barry sad.gif Wrong highlights for me to read right now- with my own exam next week !



Wouldn't worry : the stuff you've written on the forum suggests you're well prepared smile.gif
Misterioso
QUOTE(owainsutton @ Jun 28 2012, 08:58 AM) *

QUOTE(RoseRodent @ Jun 27 2012, 12:08 PM) *

... it's absolutely possible that marks would be adjusted during a moderation process.

Shifting grade boundaries for exams is one thing, but adjusting marks after the fact is completely different. What criteria would decide which candidates' marks get changed, and by how much?

Don't quote me on this, because it may have changed, and I am only giving here what an instrumental teacher told me years ago - but her understanding of the way grade exams were (are?) marked was that you would never fail by 1 mark. So if your marks added up to 99, the examiner would look at the exam as a whole and either add a mark to make it up to a pass, or deduct a mark if it wasn't quite up to standard. The only person I knew who failed by a small margin did have 98 marks, which bears out the theory.

Not sure if this is the kind of thing you meant, owainsutton, or if I've gone off at a tangent - but as RoseRodent says, marks can be adjusted during the moderation process.
Impressionist
QUOTE(flobiano @ Jun 27 2012, 12:05 PM) *

Another random thought probably linked to Deborah's observation.

Although only 9% of people fail Grade 8, only about 50-60% get merit or distinction which is roughly the same percentage of people who pass the diploma. So I think the pass mark on a diploma could also be thought of as a raising of the bar so that you need to be at least "merit" level to pass.

Hopefully that makes some sort of sense. unsure.gif


I think it does, in a way. Although having said that, having just passed my grade 8 piano there's absolutely no way I would even think about going for the first level diploma without an awful lot of work, especially on the technical side of things - I think something in the region of at least 9 months, probably closer to 18.

Judging from the scoring on my piano exam (and knowing how I actually played in the exam - I had been hoping to scrape a pass so was overjoyed to get a distinction) and comparing that back many, many years to how accurate and musical I had to be to pass my LTCL flute there's a huge jump between grade 8 and first level diplomas. It's not surprising many fail, although you'd think that if they had a teacher, they would be able to advise whether or not ready to sit. Perhaps there are quite a few out there without a teacher/mentor?
Martin.Walters
Grades are amateur ~ Diplomas are for professional development.

I can imagine many people taking their grade 8 getting through it,.. then comes the Diploma..

Its so risky, temptations and pressures.

A professional title before your name,
that impression of getting it meaning you can play almost everything (illusional)

People like to gamble for their qualifications to be the best.. but gambling brings fear,
especially if you hope to rest your career on such development.

My father says its better to do competitions as it pushes you to be the best you can be.
I would love to do a Diploma one day, but my fingers need a lot more strength/( alexander technique type stuff
to help reduce tension probably the better option)


owainsutton
QUOTE(Misterioso @ Jun 29 2012, 05:36 PM) *

QUOTE(owainsutton @ Jun 28 2012, 08:58 AM) *

QUOTE(RoseRodent @ Jun 27 2012, 12:08 PM) *

... it's absolutely possible that marks would be adjusted during a moderation process.

Shifting grade boundaries for exams is one thing, but adjusting marks after the fact is completely different. What criteria would decide which candidates' marks get changed, and by how much?

Don't quote me on this, because it may have changed, and I am only giving here what an instrumental teacher told me years ago - but her understanding of the way grade exams were (are?) marked was that you would never fail by 1 mark. So if your marks added up to 99, the examiner would look at the exam as a whole and either add a mark to make it up to a pass, or deduct a mark if it wasn't quite up to standard. The only person I knew who failed by a small margin did have 98 marks, which bears out the theory.

Not sure if this is the kind of thing you meant, owainsutton, or if I've gone off at a tangent - but as RoseRodent says, marks can be adjusted during the moderation process.

If true (and I doubt it), then this adjustment is done by the examiner at the time, and so isn't part of the moderation process. By providing the original handwritten mark sheet, the board are confirming that the marks awarded are those assigned by the person who witnessed the exam.

The only exception to this, I suppose, is when a second examiner is present in order to moderate at the time.
BerkshireMum
QUOTE(Impressionist @ Jun 29 2012, 05:45 PM) *

Judging from the scoring on my piano exam (and knowing how I actually played in the exam - I had been hoping to scrape a pass so was overjoyed to get a distinction) and comparing that back many, many years to how accurate and musical I had to be to pass my LTCL flute there's a huge jump between grade 8 and first level diplomas. It's not surprising many fail, although you'd think that if they had a teacher, they would be able to advise whether or not ready to sit. Perhaps there are quite a few out there without a teacher/mentor?

From the ABRSM statistics, only about 1000 people pa sit diploma exams (there will be some entering for other boards, but it's still not a lot), so many teachers won't have much experience of diplomas. When BerkshireSon wanted to sit DipABRSM his teacher told me she had little idea of whether he would pass, because she had never entered anyone for this diploma before and was unsure of the standard required.

We had better advice from his accompanist, though, because she had done a lot of accompanying of diploma candidates and at their first rehearsal said she thought he was at about the right level. When she came out of the exam room (leaving my son to do the viva and Quick Study) she told me he would definitely have passed the recital part, which was very reassuring!
RoseRodent
I'm wondering whether closing dates for entry have anything to do with it also. If someone is playing at and around grade standard you can put in for the exam then spend the term preparing the material. I would find it very hard to judge in May whether I'd be ready for my diploma in July, unless I were to get the whole thing so ready by May that by July I was truly sick and tired of performing it.
sbhoa
QUOTE(RoseRodent @ Jul 1 2012, 05:19 PM) *

I'm wondering whether closing dates for entry have anything to do with it also. If someone is playing at and around grade standard you can put in for the exam then spend the term preparing the material. I would find it very hard to judge in May whether I'd be ready for my diploma in July, unless I were to get the whole thing so ready by May that by July I was truly sick and tired of performing it.

I think that at this level you are into the territory where you need to live with things for a lot longer.
One consideration when choosing pieces would be 'can I live with it for long enough?'
I think that you'd have to be very experienced and confident to enter for any dip. level exam while there was still work to do one pieces to get them up to standard.
I've generally wanted to be pretty well fully prepared at point of entry for grade exams....
nicki_flute
With both of my diplomas, my pieces definitely haven't been performance ready when I entered (I think about 3 months in advance for my ATCL, and 6 months ahead for LTCL), so I just had to make sure they were by the time the exam came. I trust my teacher's judgement and am prepared to work hard to ensure they reach the standard. This was with Trinity, so the deadlines might be done differently.
andante_in_c
There is "standard" and "standard" too: my DipABRSM pupil has been trying out all the pieces/movements in various festivals and competitions over the last eighteen months, so she has had to get things to a performable standard for those. But we've been looking at them at a deeper level over the last term in order to bring the performance more to life. Learning at this level is far more of a spiral than linear, I've found.
RoseRodent
QUOTE(sbhoa @ Jul 1 2012, 05:24 PM) *
QUOTE(RoseRodent @ Jul 1 2012, 05:19 PM) *

I'm wondering whether closing dates for entry have anything to do with it also. If someone is playing at and around grade standard you can put in for the exam then spend the term preparing the material. I would find it very hard to judge in May whether I'd be ready for my diploma in July, unless I were to get the whole thing so ready by May that by July I was truly sick and tired of performing it.

I think that at this level you are into the territory where you need to live with things for a lot longer.
One consideration when choosing pieces would be 'can I live with it for long enough?'
I think that you'd have to be very experienced and confident to enter for any dip. level exam while there was still work to do one pieces to get them up to standard.
I've generally wanted to be pretty well fully prepared at point of entry for grade exams....


But as a member of this forum you come into a small contingent of people who are better informed than average. If people have done the grade system then fallen face first into the Dip because it "comes next" and have prepared for it similarly to the way they have prepared for the grades then failure is perhaps more likely?
Scooby Doo
A 50% pass rate for a professional level exam is not that bad if you look at pass rates in other professions. In a former existence, I sat a two part postgraduate diploma that had a 5% first time pass rate for each part!

blink.gif
owainsutton
QUOTE(Scooby Doo @ Jul 1 2012, 10:47 PM) *

A 50% pass rate for a professional level exam is not that bad if you look at pass rates in other professions. In a former existence, I sat a two part postgraduate diploma that had a 5% first time pass rate for each part!

The diplomas occupy something of a no-man's-land.

If they're not an academic qualification, then they're a professional one, in which case what do they qualify people for? How many Wigmore Hall recitalists have a DipABRSM?

If they're an academic qualification, then we could start wondering how they fit into the system....

Or could we just accept that they're different? They're accredited, they get brownie points in accordance with the NQF, but the most important thing is that fellow musicians can rely on these qualifications as an unvarying standard.
RoseRodent
Yes, I find them difficult as to where they fit in. The descriptions say that the DipA should be equivalent to the end of first year principal study. I have actually passed the first year of principal study in a pratical music degree and used many of the pieces from the dip syllabus, so in many ways I should be able to say I'm at the same level... but I still feel like I don't have the same standard as someone who has a dip.... why?
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.