2018 6A State

[LostChoirGuy]

I think saying that this kept Round Rock out of finals is certainly taking things far. You have no idea how much of the score was influenced by the uniforms and how much simply was a lack of execution. With only two judges for marching, it is also possible that the other judge could have been an outlier. I do not believe RR would have deserved a 35th place result in marching, but a mid 20s average from the two judges does not seem that out of place given the immensely competitive nature of the competition. Add their music scores and they definitely were a top 15 band, but I dont think they were more deserving than any of the ones that were in finals. I think we should move a bit past the complaints of this specific situation and instead seek to find solutions that can make more effective adjudication in the future.

[TxDragonDad]

The reason why the event with RR band is still a topic is that there is too much opportunity for something like this to continue to occur.   There is now enough evidence to demonstrate that the risk is no longer anecdotal and has recurred in various forms.    My involvement in the topic this year was initially welcomed with "here we go again" reactions.   That's very telling that some band feels unfairly judged too often.   

 

The UIL Marching Band Adjudication director and leadership needs to put into place protections so as to address this problem in a fair and appropriate way.   There are many viable suggestions offered.   Doing something may be only be 80% of a solution is still better than doing absolutely nothing perfectly.

 

The kids have somewhat moved on, but will never forget the time when their senior run at state prelims was so wrongfully scored by 1 judge, keeping them out of finals.   Would you?    These band kids are more equipped for success than the majority of high school students, and they will surely persevere and achieve success in their future endeavors.    However, they mostly also don't want any other kids to have to experience this issue ever again.  That's the talk I'm hearing from kids.     

[LeanderMomma]

I think saying that this kept Round Rock out of finals is certainly taking things far.

I thought it was determined that this one particular UIL judge did in fact keep RR from making finals. Forgive me if I am wrong and for spreading false information if that is in fact what I've done.

 

I'll leave it to the experts. Sorry again.

[TxDragonDad]

I think saying that this kept Round Rock out of finals is certainly taking things far. You have no idea how much of the score was influenced by the uniforms and how much simply was a lack of execution. With only two judges for marching, it is also possible that the other judge could have been an outlier. I do not believe RR would have deserved a 35th place result in marching, but a mid 20s average from the two judges does not seem that out of place given the immensely competitive nature of the competition. Add their music scores and they definitely were a top 15 band, but I dont think they were more deserving than any of the ones that were in finals. I think we should move a bit past the complaints of this specific situation and instead seek to find solutions that can make more effective adjudication in the future.

 

Their musical judge's scores average to 9.66 rank.   The other marching judge's score was a 14.   The average of the other 4 judges (including the J4's 14) was 10.75 pushing past 2 bands.     This was a grotesquely significant outlier event that caused a band to cross over the finals cut-off threshold.    

 

How far does an outlier need to be to be deemed significant?   How about when the outlier delta when compared to average of the other 4 judges greater than 66% the number of bands competing?   Is that significant enough?    27 rank places of delta in a competition of 41 bands is 66%.    

[CTJBandPops]

The reason why the event with RR band is still a topic is that there is too much opportunity for something like this to continue to occur.   There is now enough evidence to demonstrate that the risk is no longer anecdotal and has recurred in various forms.    My involvement in the topic this year was initially welcomed with "here we go again" reactions.   That's very telling that some band feels unfairly judged too often.   

 

The UIL Marching Band Adjudication director and leadership needs to put into place protections so as to address this problem in a fair and appropriate way.   There are many viable suggestions offered.   Doing something may be only be 80% of a solution is still better than doing absolutely nothing perfectly.

 

The kids have somewhat moved on, but will never forget the time when their senior run at state prelims was so wrongfully scored by 1 judge, keeping them out of finals.   Would you?    These band kids are more equipped for success than the majority of high school students, and they will surely persevere and achieve success in their future endeavors.    However, they mostly also don't want any other kids to have to experience this issue ever again.  That's the talk I'm hearing from kids.     

every band can find a judge that "kept them out" be it BOA, UIL and or any other competition that is subjectively judged. as a parent that has seen scores that I did not understand or agree with because of a wide swing, at the end of the day - it is a subjective scoring process and will remain as such - time to move on 

[CTJBandPops]

Their musical judge's scores average to 9.66 rank.   The other marching judge's score was a 14.   The average of the other 4 judges (including the J4's 14) was 10.75 pushing past 2 bands.     This was a grotesquely significant outlier event that caused a band to cross over the finals cut-off threshold.    

 

How far does an outlier need to be to be deemed significant?   How about when the outlier delta when compared to average of the other 4 judges greater than 66% the number of bands competing?   Is that significant enough?    27 rank places of delta in a competition of 41 bands is 66%.    

again - there a many programs that could claim the same variance in region, area and again at the state level - move on

[LostChoirGuy]

The music ordinals should not be considered when doing an analysis of the marching judge outlier. They were judging completely different things. Only the 14 should be considered. And with only 2 judges, either one could have been the outlier.

 

My point about the uniform not necessarily keeping them out of finals is that RR was something like 11 out of making it. We have no way of knowing that the judge placed them 11 spots lower because of the uniforms. Im sure if the judge was chosen for a state level contest, they have enough experience to notice other things as well as uniform to rank bands by, even if that is all that you heard.

[TxDragonDad]

again - there a many programs that could claim the same variance in region, area and again at the state level - move on

So, you are saying the problem is bigger than I realize and is more frequent?   Wow.  That must mean we need to drive this forward with more support.   

OK, that was a little passive aggressive by me.   I'll own that.  But, honestly, that was my first reaction to your response.   

 

So, in your opinion, outlier rankings are just part of the game and not a problem?    Kind of like playing craps in Vegas, there's always risk?  

 

Please understand, I know what competition means, and I know that champions persevere while losers blame others.   However, somewhere in the middle there are competitors who want to be fairly judged.   I'm quite certain that RR's prelim performance was worthy of a placement in the teens.  I knew they were not likely to beat the amazing shows and performances I saw of the bands which ended up in the top 8.    I think the task of ranking the bands from 9 to 19 is always daunting.   I even think that outliers on the order of a 10 rank delta (less than 25% of the competing bands) happen in EVERY event on every level.   

 

However, I doubt that more than twice that % (66%) doesn't.  If I am wrong and you are right that it does, then I believe UIL is sorely overdue in providing a solution to address adjudication.  

[Danpod]

I have a feeling that the episode on this in the coming days is going to be like an hour long!! Anyone want to sponsor a commercial break?!???

[TWHSParent]

So from a different perspective....

 

I'm going to go to the gaps of the raw scores, as that is the actual output of the judges and shows where they evaluated bands relative to each other:

 

Music (all in favor of RR):

Judge 1 gap: 123 points (6 bands in gap)

Judge 2 gap: 2 points  (no bands in gap)

Judge 3 gap: 9 points (1 band in gap)

 

Visual (all in favor of TWHS):

Judge 4 gap: 83 points (4 bands in gap)

Judge 5 gap: 103 points (17 bands in gap)

 

My view of this is that the outlier is not judge 5. Judge 5 may have had numbers management issues, which is something else to address, but the point gap in the raw score is in line.

[CTJBandPops]

I have a feeling that the episode on this in the coming days is going to be like an hour long!! Anyone want to sponsor a commercial break?!???

Danpod - here is your commercial break 

 

it is 1 am - you just left a marching band competition - you are stressed or elated - but more than anything - you are hungry -

 

But fortunately you are in Texas and there is a WhataBurger somewhere close by - 

 

so open that app and find you a whataburger or some breakfast on a bun -

 

relax - take a breath - and enjoy your food 

 

cause marching band in Texas is like the weather -  wait a few hours and it will change

 

#whataband

[Hard Core Band Fan]

If I may weigh in on the Round Rock and judge 5 issue, these are my thoughts:

 

1) Although I was disappointed for Round Rock at first, and my kids went there at one time, I know the staff and kids have moved on. It was a bad thing for them and is something they will have to live with. I am not going to tell them in any way that it doesn't matter as it certainly does matter.

 

2) I know judge 5. I know what kind of person he is/was and I know his past as a director and judge. All I am going to say is I am not the least bit surprised. I will stop at that and will say that he did what he was hired to do - give his assessment of the bands that he judged. I am in no way saying that he was correct, but UIL chose to hire him and paid him. They knew his past as well. One other thing - there were other issues with his comments that could easily be regarded as trivial, heavy-handed, and inconsistent as well.

 

3) What really bothers me is the fact that a) one judge was allowed to dictate the placement of one band far beyond what the other 4 did and the fact that in the end, almost no one outside of the affected band program cares about this, including UIL.

 

The fact that the judging system doesn't have a safeguard in place to keep a judge that is significantly different than the other 4 (20 places in this case) from having that much of an influence should be a serious concern of every band director, student, and band parent or fan. I don't care who this happens to, I don't want to see it happen to anyone! This is a flaw in the system and needs to be addressed for ALL bands who compete in this arena. If this isn't allowed to happen, then the problem no longer exists. The rules can be changed to deal with this in fair and appropriate way.

 

And that brings me to what really bugs me most of all - why this isn't likely to change. First, the people who made the finals don't care about this. It didn't affect their placement and it either didn't or never has happened to them so why go to bat for some band that they are in competition with who was less fortunate than them? No issue for them at all. Second, the other bands that didn't make the finals don't care about this. They didn't make it anyway so why would they care about one band that ended up ahead of them didn't make it? Big deal, they should just be happy to be as far up as they were, right? Third, the band directors that didn't compete this year or don't ever make it to State don't care about this. Doesn't affect them, either. Not their pain, not their problem. Fourth, UIL doesn't care. This band wasn't one of the very top bands and they know no one will even comment or complain about it. Nothing for them to do. Fifth, there is no accountability for the offending judge. No one questions him or her ever and there isn't even a discussion about it. Check written, deposited, and spent like there was never an issue at all. He or she is ready to judge again and likely will.

 

I think it is a shame that this can still happen again and could once again be allowed and forgotten. This judge will very likely be hired again and the potential for this to happen again not only exists, but is likely as well. Sadly, no one really cares enough to do anything about it. At least there are not enough of us who do and are willing to try.

[Hard Core Band Fan]

So from a different perspective....

 

I'm going to go to the gaps of the raw scores, as that is the actual output of the judges and shows where they evaluated bands relative to each other:

 

Music (all in favor of RR):

Judge 1 gap: 123 points (6 bands in gap)

Judge 2 gap: 2 points  (no bands in gap)

Judge 3 gap: 9 points (1 band in gap)

 

Visual (all in favor of TWHS):

Judge 4 gap: 83 points (4 bands in gap)

Judge 5 gap: 103 points (17 bands in gap)

 

My view of this is that the outlier is not judge 5. Judge 5 may have had numbers management issues, which is something else to address, but the point gap in the raw score is in line.

 

And how many bands did the other judges clearly knock out of the finals?

[TWHSParent]

And how many bands did the other judges clearly knock out of the finals?

 

That is a very strong statement. RR needed 21st to advance on judges preference (assuming all other things remain the same). What if that judge had them 22nd? Would we still be having this discussion? Suddenly the numbers aren't nearly as out of whack as they appear at first look now, but the result doesn't change.

 

If judge 1 would have had an ear transplant then things change too. I certainly haven't looked into the history of that judge - maybe he has a grudge against TWHS?    Or maybe he was doing the numbers management a little more correctly. Judge 5 should have given himself more room to place the bands, so his numbers management was off. That is a killer for an ordinals based system such as UIL. So I agree something needs to be done, but I think the focus should be in a different direction to address this discrepancy. Get rid of ordinals?

 

I think judge 5 underscored us as we were first off the line, thinking he needed more room above our score. That likely caused the dam of bands between TWHS and RR on his ledger, and maybe a couple he put ahead of us to try to ease the pressure. Judge 4 gave us a (IMO) more correct score and didn't have this issue, even with a similar scoring margin. Of course that is my opinion based upon my reading of the detailed results. And yes, the raw scores matter, especially when a judge paints themselves into a corner right off the bat. This could be a focal point for training judges.

 

Do people not care? Call BS on this. People care (including me), but I care more for the experience of my senior than for someone else's senior, as parents do in this situation. I'm not sorry it worked out the way it did, so I won't pretend otherwise. I have a lot of respect for Round Rock's program and gladly applaud their efforts and shows over the last 4 years and going forward, and I always feel bad for the bands that are just out of finals at any contest (because you never know when that will be you). I also think our show was better this year, and RR did not deserve to be in finals over TWHS (as I stated earlier in this thread). Frankly I haven't seen one of the posters even give a thought for the reversed outcome. Do you not care about that impact? Why is this just a one way street? Because most of the posters are from the Austin area?

[LeanderMomma]

That is a very strong statement. RR needed 21st to advance on judges preference (assuming all other things remain the same). What if that judge had them 22nd? Would we still be having this discussion? Suddenly the numbers aren't nearly as out of whack as they appear at first look now, but the result doesn't change.

 

If judge 1 would have had an ear transplant then things change too. I certainly haven't looked into the history of that judge - maybe he has a grudge against TWHS? Or maybe he was doing the numbers management a little more correctly. Judge 5 should have given himself more room to place the bands, so his numbers management was off. That is a killer for an ordinals based system such as UIL. So I agree something needs to be done, but I think the focus should be in a different direction to address this discrepancy. Get rid of ordinals?

 

I think judge 5 underscored us as we were first off the line, thinking he needed more room above our score. That likely caused the dam of bands between TWHS and RR on his ledger, and maybe a couple he put ahead of us to try to ease the pressure. Judge 4 gave us a (IMO) more correct score and didn't have this issue, even with a similar scoring margin. Of course that is my opinion based upon my reading of the detailed results. And yes, the raw scores matter, especially when a judge paints themselves into a corner right off the bat. This could be a focal point for training judges.

 

Do people not care? Call BS on this. People care (including me), but I care more for the experience of my senior than for someone else's senior, as parents do in this situation. I'm not sorry it worked out the way it did, so I won't pretend otherwise. I have a lot of respect for Round Rock's program and gladly applaud their efforts and shows over the last 4 years and going forward, and I always feel bad for the bands that are just out of finals at any contest (because you never know when that will be you). I also think our show was better this year, and RR did not deserve to be in finals over TWHS (as I stated earlier in this thread). Frankly I haven't seen one of the posters even give a thought for the reversed outcome. Do you not care about that impact? Why is this just a one way street? Because most of the posters are from the Austin area?

I was unaware that TWHS was part of this equation. I was just commenting on a judge hugely downgrading one band because he didn't like their sashes. Obviously The Woodlands deserved to be in finals. Absolutely they did. And yes, they most certainly had a better show this year than Round Rock. In fact I was hugely surprised not to see them near the top in the end.

[CTJBandPops]

That is a very strong statement. RR needed 21st to advance on judges preference (assuming all other things remain the same). What if that judge had them 22nd? Would we still be having this discussion? Suddenly the numbers aren't nearly as out of whack as they appear at first look now, but the result doesn't change.

 

If judge 1 would have had an ear transplant then things change too. I certainly haven't looked into the history of that judge - maybe he has a grudge against TWHS?    Or maybe he was doing the numbers management a little more correctly. Judge 5 should have given himself more room to place the bands, so his numbers management was off. That is a killer for an ordinals based system such as UIL. So I agree something needs to be done, but I think the focus should be in a different direction to address this discrepancy. Get rid of ordinals?

 

I think judge 5 underscored us as we were first off the line, thinking he needed more room above our score. That likely caused the dam of bands between TWHS and RR on his ledger, and maybe a couple he put ahead of us to try to ease the pressure. Judge 4 gave us a (IMO) more correct score and didn't have this issue, even with a similar scoring margin. Of course that is my opinion based upon my reading of the detailed results. And yes, the raw scores matter, especially when a judge paints themselves into a corner right off the bat. This could be a focal point for training judges.

 

Do people not care? Call BS on this. People care (including me), but I care more for the experience of my senior than for someone else's senior, as parents do in this situation. I'm not sorry it worked out the way it did, so I won't pretend otherwise. I have a lot of respect for Round Rock's program and gladly applaud their efforts and shows over the last 4 years and going forward, and I always feel bad for the bands that are just out of finals at any contest (because you never know when that will be you). I also think our show was better this year, and RR did not deserve to be in finals over TWHS (as I stated earlier in this thread). Frankly I haven't seen one of the posters even give a thought for the reversed outcome. Do you not care about that impact? Why is this just a one way street? Because most of the posters are from the Austin area?

i think it is time to move on - great points by TWHS Parent - but for every winner there is a loser and at the end of the day -while we all have opinions, none of us have been hired as judges - so let;s move on and accept what it is.

 

have a great day - 

[BlackJesus]

To be completely honest, I feel like the people saying to "move on" are part of the reason why it hasn't changed. It's not until you're one of the students that gets affected by this that you care to voice for change. Whether it be by better training, more judges, a better system, literally anything to help. Y'all simply let this happen year after year because "that's how the dice roll". I call that being complacent.

Bands shouldn't have a 20-something point spread in their ordinal scores unless their marching/music was just so much worse than the other. How is that something that's logical to y'all?

And then the "we're not the judges" argument. So every judge is fair, unbiased, and doesn't hold a grudge against bands or their directors? Please. With that argument we should never question politicians because all of them know what's best for us and never do any wrong.

 

I apologize for not wording my previous post the way I should have, about the cape. I meant that the cape was one of the reasons RR was scored lower, not the sole reason. Obviously that would be absurd.

 

I'm not trying to send a message that most of the judges are corrupt and should be prevented from ever judging again. We can all see that most are great at their jobs. And even the bad apples tend to have consistent scores with other bands. But some bands have one judge that just does not like them for one reason or another.

 

I don't know what the best solution is, I don't even know if my ideas would actually be good ideas in the end, but something NEEDS to be changed. Training, rubrics, a panel that investigates such things. Literaly anything to let the marching band community know that their concerns are being heard and addressed.

[CTJBandPops]

To be completely honest, I feel like the people saying to "move on" are part of the reason why it hasn't changed. It's not until you're one of the students that gets affected by this that you care to voice for change. Whether it be by better training, more judges, a better system, literally anything to help. Y'all simply let this happen year after year because "that's how the dice roll". I call that being complacent.

Bands shouldn't have a 20-something point spread in their ordinal scores unless their marching/music was just so much worse than the other. How is that something that's logical to y'all?

And then the "we're not the judges" argument. So every judge is fair, unbiased, and doesn't hold a grudge against bands or their directors? Please. With that argument we should never question politicians because all of them know what's best for us and never do any wrong.

 

I apologize for not wording my previous post the way I should have, about the cape. I meant that the cape was one of the reasons RR was scored lower, not the sole reason. Obviously that would be absurd.

 

I'm not trying to send a message that most of the judges are corrupt and should be prevented from ever judging again. We can all see that most are great at their jobs. And even the bad apples tend to have consistent scores with other bands. But some bands have one judge that just does not like them for one reason or another.

 

I don't know what the best solution is, I don't even know if my ideas would actually be good ideas in the end, but something NEEDS to be changed. Training, rubrics, a panel that investigates such things. Literaly anything to let the marching band community know that their concerns are being heard and addressed.

the only people that can change the scoring process are the directors - that is where the conversation needs to be had - at the end of the day, we are all sitting on the outside looking in and our opinion means nothing - so t is a matter of accepting the status quo or moving on.........

[LeanderMomma]

the only people that can change the scoring process are the directors - that is where the conversation needs to be had - at the end of the day, we are all sitting on the outside looking in and our opinion means nothing - so t is a matter of accepting the status quo or moving on.........

I feel like a lot of the directors believe it is fruitless to even go up against UIL so why even try? Its probably why some bands don't really care much about even going to state. They know they'll get a more fair shake at BOA.

 

Maybe we do need to move on, but why can we not have a conversation about it first? Educate people such as myself on what happens behind the scenes. I was certainly unaware before this season of some of the scoring issues within UIL. Maybe if we educate more people on this stuff, more parents will ask directors why it isn't changing. More voices would equal change, surely. Having a conversation is a good place to start.

[BlackJesus]

the only people that can change the scoring process are the directors - that is where the conversation needs to be had - at the end of the day, we are all sitting on the outside looking in and our opinion means nothing - so t is a matter of accepting the status quo or moving on.........

Complacency, sorry not sorry. Never been the type to accept that type of mentality.

No hard feelings.

[CTJBandPops]

I feel like a lot of the directors believe it is fruitless to even go up against UIL so why even try? Its probably why some bands don't really care much about even going to state. They know they'll get a more fair shake at BOA.

 

Maybe we do need to move on, but why can we not have a conversation about it first? Educate people such as myself on what happens behind the scenes. I was certainly unaware before this season of some of the scoring issues within UIL. Maybe if we educate more people on this stuff, more parents will ask directors why it isn't changing. More voices would equal change, surely. Having a conversation is a good place to start.

UIL is dictated and controlled by the directors - they vote routinely on the scoring process - your issues should be addressed there

[LeanderMomma]

Why do I feel as if I am missing something here? Is there a decoder ring somewhere?

[TxDragonDad]

So, I'll come back to some key points I made earlier.   

 

1) Outliers - do we think that they are a problem?

2) Outlier needs to be statistically defined, not emotionally.

3) Outlier corrective/prevention should not fully negate the judge's input, but it should also not over-correct their score, either.

 

Key points from others:

1) Directors vote, so this is on their shoulders

2) Directors of top winners and bottom losers don't care

3) Too many directors have never had it happen to them, so they don't comprehend the problem.

 

To me, the next steps are:

1) Gather real examples from history of these issues.

2) Document several solution ideas

3) Educate all TX directors on the issue, examples, and proposed solutions.

4) Use Survey Monkey to ask the director to rank the proposed solutions.

5) UIL to tell directors that a solution will be implemented, but they get to pick the which one.

 

Of course, any outlier survey responses will be hotly debated here.   LOL.

[PLC137]

I have two ways to eliminate "outlier" scores!

 

1) Have just one judge and his/her score is final - no outliers possible.

2) Have an infinite number of judges - outliers will be randomly distributed and average out among the bands.  

 

Obviously, neither of those is practical.  So we settle on a finite number of judges, greater than one.  How many?  Well, I would look at other subjectively judged competitions, like figure skating, diving, or gymnastics, which typically have about five judges.  Sometimes they throw out the high and low, sometimes they don't.  Even in head-to-head sports, like football, the referees' judgment can influence the result.  Often we blame the ref for the bad call and losing the game for us, but how about the quarterback that threw three interceptions or the point guard that missed five free throws?  

 

I think "outliers" frequently do happen in those events, so how do they cope?  If we expect all the scores to be consistent, then are we telling a judge that his/her opinion has to be be the same as the others or it doesn't count?  Might that judge have caught something the others didn't?  

 

I think the issue with throwing out data is drawing the line and agreeing on a philosophy.  Consider a three-judge system with throwing out high and low (taking the median) versus taking a average.  Do bands that have consistently good scores (say 7, 7, 7) deserve a lower ranking than bands that are all over the map (say 1, 6, 15)?  What makes the "middle" judge right and the other two "wrong?"  

 

Though the ordinal scores may amplify small differences in points, what is to say a pure points system (like BOA) are immune from bias?  I can give someone few enough points to guarantee they miss finals or a medal the same as I can flame them on ordinal rankings.  

 

As I wind down this lecture, I think it comes down to training and quality control on judges.  If one judge frequently displays a bias or uses bizarre reasoning (inconsistent with training and goals of competition) to justify a score, then we have to manage him or her out of the system - just as they do in other subjective competitions like ice dancing.  Mistakes and differences are a part of the game, but bias or negligence should be rooted out.  Unfortunately, there are victims along the way and there are always new judges coming into the system.  Hopefully we improve over time, but I don't know that a new scoring system fixes the problem.  It reminds me of work - put whatever system in place you want, but bad management can always screw it up.

 

In our lives, we've all suffered and benefited from clear official errors and biases.  The only consistent way I've found to overcome the occasional bad or unfair call is to be so good it doesn't matter.  Certainly, it never helps to get a reputation as a complainer.  

 

Thanks for reading, hope no one hates me now!

[Jane D'oh]

Key points from others:

1) Directors vote, so this is on their shoulders

2) Directors of top winners and bottom losers don't care

3) Too many directors have never had it happen to them, so they don't comprehend the problem.

 

 

Add one other point about directors' likelihood of bringing this up to UIL:

4) The directors that HAVE had this happen to their programs (the ones with the strongest motivation and the best case to make for change) aren't likely to complain either because it's perceived as whining or sour grapes, and I'm certain they feel strong pressure to take the high road and "just move on" (as we're seeing here on this thread).