PWR Rating is a JOKE

[msphi81]

Let me get this...Minnesota has lost 8 of its last 12 games and is tied for 3rd in PWR.  Moreover, three of the four wins were agains two questionable teams Minnesota State (9-14-6) and Minnesota Dulluth (11-14-5).  Great job against Michigan Tech...they lost two games at home to them...a team with only 5 other wins.  

[Robb]

Please, please, please let's not start this @#$%@# here.  Go here to post this stuff:

USCHO thread for the mathematically challenged.

[DeltaOne81]

Hey Rob, it has nothing to do with being mathematically challenged. You can understand PWR mathematically through and through, and still think that it doesn't correctly do what it was supposed to.

There are two reasons Minn is rated so high.

1) Correctly, PWR ranks based on the whole season, not just the last half-ish now. And Minn had a very good start.

2) PWR overweights Strength of Schedule, because RPI overweights S.O.S. Also as witnessed by the fact that we won tonight and our RPI still dropped very measureably. Also see NoDak for more evidence on this.

[Robb]

Of course, you are correct, DeltaOne81.  In fact, it seems to me that those who are dissatisfied with PWR tend to fall into two camps - those who don't understand it at all (and therefore think that it's black magic or some sort of conspiracy to punish/reward some teams) and those who understand it best and can provide objective reasoning as to why other rankings (KRACH, et al) do a better job of rating teams most in line with our "general sense" of who is better.

Those who fall into the second camp typically don't use words like "joke" or "questionable teams," nor do they generally cite anecdotal evidence (team A did thus-and-such, and didn't get penalized/rewarded enough).  So when I see that, I jump to the conclusion that the person doesn't understand WHY PWR does that, not just that they'd prefer a different ranking system.

My bad.

[DeltaOne81]

Robb,

And I admit that you are right that the original poster most certainly didn't have a solid enough foundation to criticize the PWR knowledgeably... most obvious evidence being the thread title. So you're right, that your response was reasonably appropriate for the level of discussion being presented.

But lets try to raise the level of dialect instead of responding in kind

This was way too intellectual of a debate to be had on USCHO. On here we might actually have a fighting chance

[Jacob '06]

And of course Minnesota is still ranked 4th in the KRACH ratings.

[KeithK]

Tha rankings still rate Minnesota very highly because they played a strong non-conference schedule which they mostly won and because their conference does very well in non-conference play.  Doing very well in your non-conf slate gives you the majority of common opponents comparison (e.g. Minnesota beats us in this category 1-0-0 to 0-1-1 based solely on the MSU games).  The WCHA's very strong record in non-conference play inflates the RPI of even the bad teams in the conference.  Losing to Michigan Tech hurts less than a loss to Yale because MTU has a solid Opponents Opponents %.  This is true both in PWR/RPI and KRACH.

I agree that it seems wrong that 12 loss Minnesota is deserving of their high ranking.  And I say this as someone who pretty much understands KRACH and doesn't have a good argument as to why KRACH is wrong.  But the system is what it is.  And keep in mind - if Minnesota keeps playing like they have this (calendar) year they won't end up a #1 seed, even if they almost certainly can't fail to get a tournament bid.

[Josh '99]

1.  I don't understand KRACH, I'll be honest.
2.  I do understand PWR.
3.  If Minnesota is ranked 4th in KRACH, then I believe KRACH rates strength of schedule too highly, just as RPI and PWR do.  But then, I think RPI should go back to 35-50-15, so my views might not reflect those of the majority.

[msphi81]

The problem with these rating systems is that they should deduct something for a loss to a lousy team.  Minnesota should be penalized for its games against Michigan Tech, Minnesota State, and Minnesota Duluth.

We had a nice win against Princeton last night, but that game did not matter in the national ratings.

Effectively top teams should rest their star players when the are playing against a weak team because the game just doesn't matter.  

Alternatively, maybe a team that has gone 4-8 and is just 13-10 in its division is the 4th best in the country.    

[KenP]

Understanding KRACH is simple...it's the derivation that's hard.  Both KRACH and RPI ratings are a combination of WIN% and SOS.  The difference is that SOS in KRACH is "more accurate" than in RPI.

WCHA teams playing well in non-conference games will inflate both their RPI and KRACH.  It may seem like a flaw, but it's the only way to compare teams that play the majority of their games in-conference.  If you want a system that awards WIN% more, you run into the risks of having cupcake AHA and CHA teams high in the rankings.

Regarding Minnesota's recent slump....the original PWR had an extra criterion for win-percentage-in-last-15-games.  That was supposed to reward "hot" teams and punish teams slumping come playoff time.  The two problems with that criterion were (a) it meant games near the end of the season were somehow more important, and (b) because most of these games tend to be in-conference, it punished teams playing in competitive conferences.

[jtwcornell91]

[Q]KenP Wrote:
Understanding KRACH is simple...it's the derivation that's hard.  Both KRACH and RPI ratings are a combination of WIN% and SOS.  The difference is that SOS in KRACH is "more accurate" than in RPI.
[/q]

Well, that, and it incorporates SOS more integrally in the ratings.  If you cook up an SOS rating and stir it in in some arbitrary way, you end up with a situation where you can hurt your ranking by beating a bad team.

[jtwcornell91]

[Q]msphi81 Wrote:
The problem with these rating systems is that they should deduct something for a loss to a lousy team.  Minnesota should be penalized for its games against Michigan Tech, Minnesota State, and Minnesota Duluth.
[/Q]

Well, I don't see why they should be punished for winning games against Minnesota State any more than we should be punished for beating Princeton.  But the losses to MTU and UMD did hurt Minnesota in KRACH.  Right now Minnesota is in 4th place in KRACH with a rating of 573.0 and an RRWP of .8018.  Go to http://slack.net/~whelan/tbrw/tbrw.cgi?2005/rankings.diy , select KRACH and "specify results" and go in and delete those three losses, and you'll see that Minnesota jumps to 3rd place, with a rating of 890.9 and and RRWP of .8546.  Change them into wins, and they're still in 3rd place, but now with a rating of 948.3 and an RRWP of .8610.  I'll let you use the form to work out the impact on RPI yourself.

[Q]We had a nice win against Princeton last night, but that game did not matter in the national ratings.[/Q]

We were overwhelmingly favored to beat Princeton last night.  If we hadn't beaten them, that would be news.  We shouldn't be hurt by the win, like we are in RPI, but I don't see the problem with the improvement in our standing for beating a team much weaker than us being only marginal, like it is in KRACH.

[Q]Effectively top teams should rest their star players when the are playing against a weak team because the game just doesn't matter.[/Q]

No!  Winning those games won't help you much, but losing them will hurt you a lot.  It's the tradeoff for playing a weaker opponent you're more likely to beat; the payoff is commensurate with the odds.

[Q]Alternatively, maybe a team that has gone 4-8 and is just 13-10 in its division is the 4th best in the country.[/q]

Have you looked at their opponents? http://slack.net/~whelan/tbrw/tbrw.cgi?2005/current/Kcrit.Mn 11 of their 32 games were against Denver, CC, and Wisconsin, all ranked ahead of them in KRACH.  That makes their ranking fairly consistent with their 20-12 overall record.

[Ken \'70]

[Q]KeithK Wrote:

  And keep in mind - if Minnesota keeps playing like they have this (calendar) year they won't end up a #1 seed, even if they almost certainly can't fail to get a tournament bid.[/q]


Looking at the individual PWR comparisons for MN it's hard to see how they lose more than 2 comparisons from here out, MI and BU being the only visible candidates.  

To lose MI they need AA to become TUC and they have to be no better than .500 in their final series against Mich. Tech.  

To lose BU they need to lose the RPI, which is doable, and have BU do no worse than a win and tie against MA next weekend.  

But the key comparison is MI.  Losing only BU will still put MN in a tie wth MI (if MI itself doesn't lose anymore) and MN will finish ahead based on the tie breaker.  

For those of you thinking Cornell might finish ahead of MN, forget it.  Cornell will lose the BC comparison it's now winning, will lose the MI comparison tonight if MI beats NO, can't flip MN itself, and is at risk of losing the OS comparison once OS plays MS and SLU drops from TUC (under the bonus scenario, SLU is already out of TUC without bonuses).

A final PWR with MN 4 and Cornell 5 is a nightmare for Cornell fans.   Almost any scenario in which MN finishes 5 is a nightmare for the committee.

[jkahn]

Why PWR is a joke (note: this is not an anti-Minnesota article, as I am a KRACH believer):
1) RPI, a very important factor in PWR, is not even internally consistent.  RPI is 25% your winning percentage, 50% your opponents' winning %, and 25 % your opponents' opponents winning %.  That last 75% is in there to factor in your strength of schedule.  However, if RPI is really a better measure of a team's strength than winning %, why isn't that final 75% similarly divided into 3 parts,  i.e., if a team had Cornell on its schedule,  Cornell's strength should be factored in as 25% Cornell's record, 50% Cornell's opponent's record and 25% Cornells opp-opp record to be consistent (not 2/3 winning % and 1/3 opp. win %).  I don't believe this would be right either, but just want to point out that opponent's record is very overweighted in the current system, and if RPI is supposedly a better measure than winning percentage, than something that resembles opponent's RPI should be the strength of schedule factor.
2) Here's a hypothethical.  One team plays every team in the country and has a .750 record.  Another team plays only the ten teams that have .750 records and ends up with a .500 record.  Which team is better?  It seems that the first is a .750 team and the second is just as good as the average .750 team.  I believe they are equal.  Yet RPI would rank the second team much higher.
[Note: there is a counter-argument here that since conference play forces percentages within a conference toward .500, that SOS need a higher weight.]
3) Is it really better to lose to Michigan Tech and beat Denver, than the opposite?  I think if you go 1-1 it shouldn't matter whether you've beaten the strong team and lost to the weak or vice versa.  The TUC concept however, makes losses to weak teams count less than losses to strong teams.
4) The whole concept of the TUC cutoff introduces strange variations as opponents go in and out of TUC-land.  In the interest of brevity, I won't give examples - but merely indicate that a step function (TUC, non-TUC) causes all sorts of aberrations.
5) Any system where there is a possibility that you can look back at the end of the season and say that a team would have been better off tying or losing a game than winning it is not a good system.

In addition to the above, since non-conference records of teams in your conference plays a very significant role in strength of schedule (75% of the RPI), the ECAC is at a decided disadvantage because many of its teams are playing these games much earlier in the season than the opponents.  When we opened at OSU in '02-'03, I think it was their 7th game.

[DeltaOne81]

 [Q]Ken '70 Wrote:
For those of you thinking Cornell might finish ahead of MN, forget it.  Cornell will lose the BC comparison it's now winning, will lose the MI comparison tonight if MI beats NO, can't flip MN itself, and is at risk of losing the OS comparison once OS plays MS and SLU drops from TUC (under the bonus scenario, SLU is already out of TUC without bonuses).

A final PWR with MN 4 and Cornell 5 is a nightmare for Cornell fans.   Almost any scenario in which MN finishes 5 is a nightmare for the committee.[/Q]
I don't know if we can finish ahead of them, but we *can* win our comparison against MN. All we need to do it take TUC. We can't do it ourself, but we can with a little help from Minn. We can get one more TUC win against SLU, assuming they can keep themselves a TUC.

Minn, meanwhile, can help make UAA a TUC, especially if they lose to them again tonight, and that would add 2 loses (hypothetically) to their TUC column. They also play SCSU, who just  beat CC tonight, and with the way Minn has been playing, they could easily lose those. Are at least one. At 2 loses to UAA, and 2 to SCSU, or even 1-3 total, plus a win for us, and I believe we could flip the comparison with Minn.

Keeping SLU a TUC might be harder than we'd like. There's no way to know if the committee ads RPI bonuses to everyone, or if they might just do it for TUCs - either intentionally, or just by neglect, thinking it wouldn't matter when it really would.