RPI primer (brackets for dummies): Why #2/#3 Cornell may seed lower

[ugarte]

[Q]Beeeej Wrote:

 What I got out of Newman's post:

Blah blah blah blah blah blah
Blah blah blah blah blah
Blah blah blah blah blah

Can't you talk about something really useful, like the motion practice aspects or fourth amendment implications of KRACH, or something?    

Beeeej[/q]This is unacceptable. Finals are months away and you are a 3L.

That said, John, please help translate what Newman said...

[KeithK]

Seems like an awful lot of variance, esp. at the top (CC could have a 4k rating? Wow!) Maybe we need to lower the confidence level somewhat, at least to 90%.  I guess it does sort of follow though.  If a team like Wisconsin (#4) can manage to tie Yale (#48) then there certainly is a lot of variance in the actual play of the teams, which should be reflected in the variance of the rankings.

[ninian '72]

The confidence interval of an estimate is that range of values that we cannot say have a statistically significant difference from the estimate.  (In this case using the fairly common .05 level of significance.  Hence 1 - .05 =.95 or 95%.)  Newman says that the top 15 teams are within the confidence interval for the score of the #1 rated team, which means that none of their KRACH ratings are statistically different from that for the #1 team.  So, setting the number of teams participating in the NCAA tournament at 16 is good practice, since it's close to the number of top teams whose KRACH ratings are not significantly different.  Hopefully this helps, and you can take it from there.

[KeithK]

[q]So, setting the number of teams participating in the NCAA tournament at 16 is good practice, since it's close to the number of top teams whose KRACH ratings are not significantly different.[/q]I wouldn't be too quick to draw that conclusion.  This year's results may show a match between number of tournament bids and statistically comparable #1 teams according to KRACH.  But it remains to be seen whether this pattern holds for earlier years.  

[Beeeej]

[Q]ugarte Wrote:This is unacceptable. Finals are months away and you are a 3L.
[/q]

Yeah, but I'm in the thick of the interview process.

Beeeej

[elliotb]

[Q]Newman Wrote:

> Teams 1-15 in KRACH have a statistical claim on being #1 at a 95% confidence level (which, ignoring the fact the KRACH has little to do with tournament selection, makes having a 16 team tournament rather an auspicious size).[/q]

Are you making that claim based on the fact that the confidence intervals for the top 15 teams all cover CC's actual KRACH rating? Or is it based on the analysis in your spreadsheet where everything was done relative to CC? (Perhaps those are equivalent, although I suspect not.)

I must say, I'm surprised the intervals are so wide. I guess that explains why there's so much arguing over rankings.

- Elliot

[Newman]

[Q]KeithK Wrote:

 [Q2]So, setting the number of teams participating in the NCAA tournament at 16 is good practice, since it's close to the number of top teams whose KRACH ratings are not significantly different.[/Q]
I wouldn't be too quick to draw that conclusion.  This year's results may show a match between number of tournament bids and statistically comparable #1 teams according to KRACH.  But it remains to be seen whether this pattern holds for earlier years.  [/q]

What I meant was that it coincidentally happened to be a close match this year, not that 16 is a good number all around every year.  Also, no teams from AH or CHA are in that group, so at least one of those 15 teams will be shut out of the NCAAs.

[Newman]

[Q]elliotb Wrote:

 [Q2]Newman Wrote:

> Teams 1-15 in KRACH have a statistical claim on being #1 at a 95% confidence level (which, ignoring the fact the KRACH has little to do with tournament selection, makes having a 16 team tournament rather an auspicious size).[/Q]
Are you making that claim based on the fact that the confidence intervals for the top 15 teams all cover CC's actual KRACH rating? Or is it based on the analysis in your spreadsheet where everything was done relative to CC? (Perhaps those are equivalent, although I suspect not.)

- Elliot[/q]

I'm making that claim based on CI's covering CC's rating when CC is the reference team. Comparing two teams ratings when one team isn't the reference school is more complex, and requires incorporating the covariance.

Also, really this is just showing the chance of a team having a KRACH higher than Colorado College. To be #1 they'd have to have a rating higher than all teams, which is a much more complex calculation I'm not going to try to do.

[billhoward]

It's possible they're BS'ing us, but it's also possible there's some underlying beauty and symmetry if you apply enough math to all this.  

But the margin of error is always going to be there and it's as least as great as the sum of all posts and crossbars hit plus Zambonis making double passes across the ice and leaving it wet, and players coming out of the penalty box just as the puck is passed to center ice.

[RichH]

[Q]elliotb Wrote:

I must say, I'm surprised the intervals are so wide. I guess that explains why there's so much arguing over rankings.[/q]

Well, that's just the nature of KRACH: it asymptotically blows up with records closer and closer to perfection.  Cornell's 1970 KRACH was infinite.  So a small variation on the horizontal (record) axis creates huge variations in KRACH possibilites, thus the large variances (or error bars, if you will) in the KRACH, especially with only a 30-40 game sample.  

It'd probably be more managable if you do the same 95% confidence interval calculations on the RRWP (Round-Robin Winning Percentage) which is still tied to the KRACH, but bounded between 0.000 and 1.000, by definition.  Even then, because the teams at the top are so close (the value of flipping one win to a non-win or vice-versa is greater than the SOS difference for many of the teams at the top), you'll still see a large amount of overlap.  Only with prettier numbers.  :-)

[ninian '72]

[Q]Newman Wrote:

 [Q2]KeithK Wrote:

 [Q2]So, setting the number of teams participating in the NCAA tournament at 16 is good practice, since it's close to the number of top teams whose KRACH ratings are not significantly different.[/Q]
I wouldn't be too quick to draw that conclusion.  This year's results may show a match between number of tournament bids and statistically comparable #1 teams according to KRACH.  But it remains to be seen whether this pattern holds for earlier years.  [/Q]
What I meant was that it coincidentally happened to be a close match this year, not that 16 is a good number all around every year.  Also, no teams from AH or CHA are in that group, so at least one of those 15 teams will be shut out of the NCAAs.[/q]

Agreed.  It works this year (so far), but no guarantees for other years.

[ninian '72]

[Q]billhoward Wrote:

 It's possible they're BS'ing us, but it's also possible there's some underlying beauty and symmetry if you apply enough math to all this.[/q]

BS??  :-O  There are enough good statisticians and mathematicians on this board to keep us each of us honest.  

[q]But the margin of error is always going to be there and it's as least as great as the sum of all posts and crossbars hit plus Zambonis making double passes across the ice and leaving it wet, and players coming out of the penalty box just as the puck is passed to center ice. [/q]

That was part of my point in the original post, although I restricted it to goal differentials.  No matter what method is used, these are only sophisticated indices that attempt to find a common denominator to compare the whole field of Division I teams.  In the process, a lot of the contextual variability, whether it's ice surface, unlucky bounces, injuries to key players, defensive strategy, or a vocal home crowd, is not taken into account.  As such, these indices are never going to be completely satisfactory predictors.  I'm grateful for that.  Wouldn't be much fun to be a hockey fan, if there weren't some sort of suspense about the outcome of any particular game, would it?

[Newman]

[Q]billhoward Wrote:
But the margin of error is always going to be there and it's as least as great as the sum of all posts and crossbars hit plus Zambonis making double passes across the ice and leaving it wet, and players coming out of the penalty box just as the puck is passed to center ice. [/q]

Indeed; I neglected to mention in my post (although it's in the excel file if you downloaded it) that the process for finding the variance also tells you, on average, what percentage of the variance in game outcomes is explained by KRACH. In other words, how much of a factor the different KRACH ratings are in determining who wins. This year it's 19% across the whole season so far, and a lot of that is from the expected blowouts. So in games between closer matched teams, the outcome is well over 80% based on the other factors (Zambonis and crossbars and baaaa-ad calls, oh my!)

[billhoward]

>>> Indeed; I neglected to mention in my post (although it's in the excel file if you downloaded it) that the process for finding the variance also tells you, on average, what percentage of the variance in game outcomes is explained by KRACH. In other words, how much of a factor the different KRACH ratings are in determining who wins. This year it's 19% across the whole season so far, and a lot of that is from the expected blowouts. So in games between closer matched teams, the outcome is well over 80% based on the other factors (Zambonis and crossbars and baaaa-ad calls, oh my!)

(Is Newman your name or do you like Seinfeld characters?)

This post actually made sense. I figured if I read long enough (all the posts), I'd find a pony in there somewhere. So you're saying (forgive me for the journalist's trick of trying to reduce a complex argument to something simple enough for both scribe and audience to fathom) that in four of every five games between closely matched teams, the outcome is due to other factors such as blown calls and the puck bouncing this way not that off the pipe? And other other fifth is the right team actually winning for the right reason eg having modestly more measurable skill?

If 80% of victories are due to "other" factors then Cornell's unbeaten string is all the more impressive because the odds should have caught up with us somewhere, and in the entire new year it only happened 1-1/2 times, at Harvard (last loss before the streak) and vs. Colgate (tie).

I wonder if one could determine that a team playing a solid defensive game that results in 2-0 and 2-1 scores has more, better, and/or more consistent outcomes than offensive flyers who win 8-5, 7-3, and occasionally lose 6-5? If so, then there's solid math behind Schafer's madness.

[KeithK]

He said in games against closely matched teams 80% of the result is due to "other factors".  During our recent streak, a number of games were against teams that were not close matched (according to KRACH) to Cornell.

[q](Is Newman your name or do you like Seinfeld characters?) [/q]Bill, you must really have a different Cornell hockey fandom frame of reference than I do if you question whether someone is really named Newman. :-)