2018 6A State

[Nny14]

  On 11/11/2018 at 6:16 PM, scoobydoo said:

Agreed. I wonder if RR's director ever addressed his concern about this judge to UIL?

He's not the type to place blame on judges, but I don't think a complaint would be ridiculous at this point. Imagine if this person had to judge Broken Arrow this year.

[LeanderMomma]

  On 11/11/2018 at 7:05 PM, Nny14 said:

He's not the type to place blame on judges, but I don't think a complaint would be ridiculous at this point. Imagine if this person had to judge Broken Arrow this year.

I was thinking the same thing and actually it’s been said that the skirts did hurt BA this year because some judges couldn’t see their legs. I don’t see how leg capes affect that in the same way though. Round Rock’s uniform is nothing like Broken Arrow and their skirts.

[Jane D'oh]

Here's another idea for UIL:

 

What about appointing a "Chief Judge," whose job is to judge the judges. He or she would review the placements before they're released, and if there are any real standouts (like maybe a 10+ ordinal difference among judges in one category), the Chief Judge would approach the judge(s) to get justification for their decisions. If they say it was because they saw two kids at the back of the 10-yd line doing the Chicken Dance during the ballad, okay. The other marching judge probably couldn't see that, which would explain the huge difference. The score stands. If they give a reason that's not within the UIL judging criteria, like not liking the uniforms, the Chief Judge can overrule. Maybe watch video replay to judge for themselves. A band should be able to march wearing trash bags and get the same fair look as all the other bands in a UIL competition. (Not that Round Rock's uniforms are trash bags -- in my opinion, they're the best-looking uniforms on the field, bar none.) 

 

That would be a less expensive solution than going to 9 judges, and it would add a layer of accountability that might set a lot of minds at ease in the future. Of course, there's always the risk that the Chief Judge would be biased as well, but like a newspaper hiring an ombudsman, it behooves the organization to police itself and provide transparency if they want to maintain the trust and respect of the people they serve.

[BlackJesus]

  On 11/11/2018 at 9:25 PM, Jane D'oh said:

Here's another idea for UIL:

 

What about appointing a "Chief Judge," whose job is to judge the judges. He or she would review the placements before they're released, and if there are any real standouts (like maybe a 10+ ordinal difference among judges in one category), the Chief Judge would approach the judge(s) to get justification for their decisions. If they say it was because they saw two kids at the back of the 10-yd line doing the Chicken Dance during the ballad, okay. The other marching judge probably couldn't see that, which would explain the huge difference. The score stands. If they give a reason that's not within the UIL judging criteria, like not liking the uniforms, the Chief Judge can overrule. Maybe watch video replay to judge for themselves. A band should be able to march wearing trash bags and get the same fair look as all the other bands in a UIL competition. (Not that Round Rock's uniforms are trash bags -- in my opinion, they're the best-looking uniforms on the field, bar none.)

 

That would be a less expensive solution than going to 9 judges, and it would add a layer of accountability that might set a lot of minds at ease in the future. Of course, there's always the risk that the Chief Judge would be biased as well, but like a newspaper hiring an ombudsman, it behooves the organization to police itself and provide transparency if they want to maintain the trust and respect of the people they serve.

I like the idea, but feel like the judges might lie about why they scored them so much lower. But then again, the judges might know that they can't screw anyone over or else be questioned, therefore scoring fairly. I've also always been in favor of investigating judges for stuff like this. You can't tell me every single judge is an angel.

[Samuel Culper]

If the chief judge has a record with him/her of each judge's history of outlier scores, that would help as well.

[1998-2018]

Reviewing ordinal scores/placements should be triggered by a percentage deviation. An ordinal point swing from 1 to 10 as happened at Area B or 1 to 9 at another Area is much more statistically significant than a swing from 74 to 84 or 100 to 110 as could theoretically happen at BOA SA or Indy Grand Nationals.

 

Percentage statistics are almost always more informative, revealing, and helpful than counting statistics.

[1998-2018]

We should also keep in mind that a low outlier might be accompanied by a high outlier from the same judge. Or a high outlier can also happen across different judges just like the lows people tend to notice more often. Both ends of the spectrum need to be actively monitored.

[TxDragonDad]

  On 11/12/2018 at 5:25 AM, 1998-2018 said:

We should also keep in mind that a low outlier might be accompanied by a high outlier from the same judge. Or a high outlier can also happen across different judges just like the lows people tend to notice more often. Both ends of the spectrum need to be actively monitored.

 

There was but not to the extent of the detrimental impact to RR.    J5's rating of John Horn was overly inflated when compared to the average of all the other 4 judges.    If you look back to my original post this year, I provide a write up of the significance of the outlier impact and J5's impact.  

[TxDragonDad]

Regarding the leg "skirts" of RR Dragon Band - they are more like large sashes.   For those who would like to study them take a look at this picture.   Personally, I do not see how they could possibly obscure judging of marching.   RR band does not do any high knee moves.   The sash is a fixed size/length and can vary slightly in appearance on different sized kids.  It covers less than 1/3 the waistline circumference on most students, but it varies by waist size.   (Larger kids are less covered, super thin kids are more covered.)  The sash is roughly knee length, again appearing less on some and slightly more on others.   When moving in a large stride, the sash is pulled away from the torso and the legs easily appear separate and moving independently.   When hitting "place" or forms, the sash is irrelevant as the feet, torso and Shako plumes can easily be used to judge spacing and formation.

 

[LeanderMomma]

  On 11/12/2018 at 5:25 PM, TxDragonDad said:

Regarding the leg "skirts" of RR Dragon Band - they are more like large sashes. For those who would like to study them take a look at this picture. Personally, I do not see how they could possibly obscure judging of marching. RR band does not do any high knee moves. The sash is a fixed size/length and can vary slightly in appearance on different sized kids. It covers less than 1/3 the waistline circumference on most students, but it varies by waist size. (Larger kids are less covered, super thin kids are more covered.) The sash is roughly knee length, again appearing less on some and slightly more on others. When moving in a large stride, the sash is pulled away from the torso and the legs easily appear separate and moving independently. When hitting "place" or forms, the sash is irrelevant as the feet, torso and Shako plumes can easily be used to judge spacing and formation.

 

RRDragon Leg Sash.png

Yes I agree 100%. That sash does nothing to impede a judge’s view of how RR is marching. I would surely like to hear from this judge what exactly his gripe is regarding the sash. I bet he abhors Broken Arrow this season! Or maybe he just has something against Round Rock.

[CTJBandPops]

  On 11/12/2018 at 5:29 PM, LeanderMomma said:

Yes I agree 100%. That sash does nothing to impede a judge’s view of how RR is marching. I would surely like to hear from this judge what exactly his gripe is regarding the sash. I bet he abhors Broken Arrow this season! Or maybe he just has something against Round Rock.

i think we are getting hung up in the weeds - there are many things that may play into the judges interpretation of the marching. there are 5 judges in UIL and at the end of the day, it is what it is.

 

we can all agree that their needs to be a change in the UIL judging process - but I think that stating that a specific judge had a vendetta against Round Rock is a bit out there and does no one any good - I would assume that the kids have all moved on and that we should do the same.  

[Mash]

  On 11/12/2018 at 5:25 PM, TxDragonDad said:

Regarding the leg "skirts" of RR Dragon Band - they are more like large sashes.   For those who would like to study them take a look at this picture.   Personally, I do not see how they could possibly obscure judging of marching.   RR band does not do any high knee moves.   The sash is a fixed size/length and can vary slightly in appearance on different sized kids.  It covers less than 1/3 the waistline circumference on most students, but it varies by waist size.   (Larger kids are less covered, super thin kids are more covered.)  The sash is roughly knee length, again appearing less on some and slightly more on others.   When moving in a large stride, the sash is pulled away from the torso and the legs easily appear separate and moving independently.   When hitting "place" or forms, the sash is irrelevant as the feet, torso and Shako plumes can easily be used to judge spacing and formation.

 

RRDragon Leg Sash.png

Crazy how that would affect judging.

[LeanderMomma]

  On 11/12/2018 at 5:37 PM, banddad84 said:

i think we are getting hung up in the weeds - there are many things that may play into the judges interpretation of the marching. there are 5 judges in UIL and at the end of the day, it is what it is.

 

we can all agree that their needs to be a change in the UIL judging process - but I think that stating that a specific judge had a vendetta against Round Rock is a bit out there and does no one any good - I would assume that the kids have all moved on and that we should do the same.

Normally I would agree with you, but this situation has repeated itself twice, keeping Round Rock out of finals. If this was a crime scene and I were a cop, that would definitely make me look a bit harder into it. I also feel that it validates the need for dropping lowest and highest scores.

[CTJBandPops]

  On 11/12/2018 at 5:37 PM, Mash said:

Crazy how that would affect judging.

lets stop for a minute and review a few things that have been discussed on these posts in the past as well as conversations I have had and consider it as part of the overall conversation. There are some known dynamics that play into the overall visual or Marching response for a band program - these are factors that play into both UIL and BOA competitions.

• What is the most common uniform color??? - it is black - if you watched both the BOA and UIL events in the past week you will see that Black is the most common color and if you are not familiar with a particular program - you cannot tell many of them apart - WHY -  against a green turf, Black hides the dirt - white stands out and will accent the marching transitions or lack their of. 
• Headgear - we have seen a number of programs that went without headgear/hats as it accents the body movement or lack of such and can impact the visual score or in the case of UIL - the marching/maneuvering scores
• Plumes - in addition to headgear - the plume highlights the body movements - a plume that stands out in white or a bright color can have an impact. Some of you may have noticed that there were a number of programs that changed plumes, eliminated plumes, or even eliminated head wear between BOA and UIL this past week
• Leg Sashes - like everything else that defines the body movement - this could be an issue - in UIL more so than BOA IMO

every director understands the visual impact that they put on the field and it is on them to address it accordingly

 

just my thoughts

[LostChoirGuy]

I think saying that this kept Round Rock out of finals is certainly taking things far. You have no idea how much of the score was influenced by the uniforms and how much simply was a lack of execution. With only two judges for marching, it is also possible that the other judge could have been an outlier. I do not believe RR would have deserved a 35th place result in marching, but a mid 20s average from the two judges does not seem that out of place given the immensely competitive nature of the competition. Add their music scores and they definitely were a top 15 band, but I dont think they were more deserving than any of the ones that were in finals. I think we should move a bit past the complaints of this specific situation and instead seek to find solutions that can make more effective adjudication in the future.

[TxDragonDad]

The reason why the event with RR band is still a topic is that there is too much opportunity for something like this to continue to occur.   There is now enough evidence to demonstrate that the risk is no longer anecdotal and has recurred in various forms.    My involvement in the topic this year was initially welcomed with "here we go again" reactions.   That's very telling that some band feels unfairly judged too often.   

 

The UIL Marching Band Adjudication director and leadership needs to put into place protections so as to address this problem in a fair and appropriate way.   There are many viable suggestions offered.   Doing something may be only be 80% of a solution is still better than doing absolutely nothing perfectly.

 

The kids have somewhat moved on, but will never forget the time when their senior run at state prelims was so wrongfully scored by 1 judge, keeping them out of finals.   Would you?    These band kids are more equipped for success than the majority of high school students, and they will surely persevere and achieve success in their future endeavors.    However, they mostly also don't want any other kids to have to experience this issue ever again.  That's the talk I'm hearing from kids.     

[LeanderMomma]

  On 11/12/2018 at 6:20 PM, LostChoirGuy said:

I think saying that this kept Round Rock out of finals is certainly taking things far.

I thought it was determined that this one particular UIL judge did in fact keep RR from making finals. Forgive me if I am wrong and for spreading false information if that is in fact what I've done.

 

I'll leave it to the experts. Sorry again.

[TxDragonDad]

  On 11/12/2018 at 6:20 PM, LostChoirGuy said:

I think saying that this kept Round Rock out of finals is certainly taking things far. You have no idea how much of the score was influenced by the uniforms and how much simply was a lack of execution. With only two judges for marching, it is also possible that the other judge could have been an outlier. I do not believe RR would have deserved a 35th place result in marching, but a mid 20s average from the two judges does not seem that out of place given the immensely competitive nature of the competition. Add their music scores and they definitely were a top 15 band, but I dont think they were more deserving than any of the ones that were in finals. I think we should move a bit past the complaints of this specific situation and instead seek to find solutions that can make more effective adjudication in the future.

 

Their musical judge's scores average to 9.66 rank.   The other marching judge's score was a 14.   The average of the other 4 judges (including the J4's 14) was 10.75 pushing past 2 bands.     This was a grotesquely significant outlier event that caused a band to cross over the finals cut-off threshold.    

 

How far does an outlier need to be to be deemed significant?   How about when the outlier delta when compared to average of the other 4 judges greater than 66% the number of bands competing?   Is that significant enough?    27 rank places of delta in a competition of 41 bands is 66%.    

[CTJBandPops]

  On 11/12/2018 at 6:24 PM, TxDragonDad said:

The reason why the event with RR band is still a topic is that there is too much opportunity for something like this to continue to occur.   There is now enough evidence to demonstrate that the risk is no longer anecdotal and has recurred in various forms.    My involvement in the topic this year was initially welcomed with "here we go again" reactions.   That's very telling that some band feels unfairly judged too often.   

 

The UIL Marching Band Adjudication director and leadership needs to put into place protections so as to address this problem in a fair and appropriate way.   There are many viable suggestions offered.   Doing something may be only be 80% of a solution is still better than doing absolutely nothing perfectly.

 

The kids have somewhat moved on, but will never forget the time when their senior run at state prelims was so wrongfully scored by 1 judge, keeping them out of finals.   Would you?    These band kids are more equipped for success than the majority of high school students, and they will surely persevere and achieve success in their future endeavors.    However, they mostly also don't want any other kids to have to experience this issue ever again.  That's the talk I'm hearing from kids.     

every band can find a judge that "kept them out" be it BOA, UIL and or any other competition that is subjectively judged. as a parent that has seen scores that I did not understand or agree with because of a wide swing, at the end of the day - it is a subjective scoring process and will remain as such - time to move on 

[CTJBandPops]

  On 11/12/2018 at 6:35 PM, TxDragonDad said:

Their musical judge's scores average to 9.66 rank.   The other marching judge's score was a 14.   The average of the other 4 judges (including the J4's 14) was 10.75 pushing past 2 bands.     This was a grotesquely significant outlier event that caused a band to cross over the finals cut-off threshold.    

 

How far does an outlier need to be to be deemed significant?   How about when the outlier delta when compared to average of the other 4 judges greater than 66% the number of bands competing?   Is that significant enough?    27 rank places of delta in a competition of 41 bands is 66%.    

again - there a many programs that could claim the same variance in region, area and again at the state level - move on

[LostChoirGuy]

The music ordinals should not be considered when doing an analysis of the marching judge outlier. They were judging completely different things. Only the 14 should be considered. And with only 2 judges, either one could have been the outlier.

 

My point about the uniform not necessarily keeping them out of finals is that RR was something like 11 out of making it. We have no way of knowing that the judge placed them 11 spots lower because of the uniforms. Im sure if the judge was chosen for a state level contest, they have enough experience to notice other things as well as uniform to rank bands by, even if that is all that you heard.

[TxDragonDad]

  On 11/12/2018 at 6:40 PM, banddad84 said:

again - there a many programs that could claim the same variance in region, area and again at the state level - move on

So, you are saying the problem is bigger than I realize and is more frequent?   Wow.  That must mean we need to drive this forward with more support.   

OK, that was a little passive aggressive by me.   I'll own that.  But, honestly, that was my first reaction to your response.   

 

So, in your opinion, outlier rankings are just part of the game and not a problem?    Kind of like playing craps in Vegas, there's always risk?  

 

Please understand, I know what competition means, and I know that champions persevere while losers blame others.   However, somewhere in the middle there are competitors who want to be fairly judged.   I'm quite certain that RR's prelim performance was worthy of a placement in the teens.  I knew they were not likely to beat the amazing shows and performances I saw of the bands which ended up in the top 8.    I think the task of ranking the bands from 9 to 19 is always daunting.   I even think that outliers on the order of a 10 rank delta (less than 25% of the competing bands) happen in EVERY event on every level.   

 

However, I doubt that more than twice that % (66%) doesn't.  If I am wrong and you are right that it does, then I believe UIL is sorely overdue in providing a solution to address adjudication.  

[Danpod]

I have a feeling that the episode on this in the coming days is going to be like an hour long!! Anyone want to sponsor a commercial break?!???

[TWHSParent]

So from a different perspective....

 

I'm going to go to the gaps of the raw scores, as that is the actual output of the judges and shows where they evaluated bands relative to each other:

 

Music (all in favor of RR):

Judge 1 gap: 123 points (6 bands in gap)

Judge 2 gap: 2 points  (no bands in gap)

Judge 3 gap: 9 points (1 band in gap)

 

Visual (all in favor of TWHS):

Judge 4 gap: 83 points (4 bands in gap)

Judge 5 gap: 103 points (17 bands in gap)

 

My view of this is that the outlier is not judge 5. Judge 5 may have had numbers management issues, which is something else to address, but the point gap in the raw score is in line.

[CTJBandPops]

  On 11/12/2018 at 7:33 PM, Danpod said:

I have a feeling that the episode on this in the coming days is going to be like an hour long!! Anyone want to sponsor a commercial break?!???

Danpod - here is your commercial break 

 

it is 1 am - you just left a marching band competition - you are stressed or elated - but more than anything - you are hungry -

 

But fortunately you are in Texas and there is a WhataBurger somewhere close by - 

 

so open that app and find you a whataburger or some breakfast on a bun -

 

relax - take a breath - and enjoy your food 

 

cause marching band in Texas is like the weather -  wait a few hours and it will change

 

#whataband