PWR Rank vs. KRACH Rank
Posted by Scersk '97
PWR Rank vs. KRACH Rank
Posted by: Scersk '97 (---.dsl.chcgil.ameritech.net)
Date: February 12, 2005 05:59PM
Oh, and since I mentioned it in the other thread, I'll post my simplistic comparison of the two ranking schemes at this moment. For the record, I'm a huge believer in KRACH as a more accurate representation of team strength. So I feel, for example, that Wisconsin being undervalued in PWR is really going to screw someone in the tournament:
Well, someone besides just Wisconsin. To my mind, there's just no way to deny that the WCHA has been dominant as a conference this year, and the way that the RPI is set up absolutely screws them for beating up on each other. (Something we usually complain about in the ECAC.)
Oh, and way to go Big Green!
As of 2/12/05 4:50 PM CST: PWR KRACH Delta ===================================================== 12 Dartmouth 24 Dartmouth -12 27 Alabama-Huntsville 34 Alabama-Huntsville -7 24 Western Michigan 31 Western Michigan -7 26 Bemidji State 32 Bemidji State -6 20 Vermont 26 Vermont -6 7 Harvard 12 Harvard -5 2 Boston College 5 Boston College -3 22 Brown 25 Brown -3 4 Cornell 7 Cornell -3 20 Nebraska-Omaha 23 Nebraska-Omaha -3 19 Northeastern 21 Northeastern -2 15 Colgate 16 Colgate -1 9 Ohio State 10 Ohio State -1 2 Colorado College 2 Colorado College 0 1 Denver 1 Denver 0 6 Michigan 6 Michigan 0 17 Michigan State 17 Michigan State 0 4 Minnesota 4 Minnesota 0 11 North Dakota 11 North Dakota 0 15 Northern Michigan 15 Northern Michigan 0 28 Bowling Green 27 Bowling Green 1 14 Mass.-Lowell 13 Mass.-Lowell 1 10 Boston University 8 Boston University 2 17 Maine 14 Maine 3 12 New Hampshire 9 New Hampshire 3 22 Minnesota State 18 Minnesota State 4 24 St. Cloud State 19 St. Cloud State 5 8 Wisconsin 3 Wisconsin 5
Well, someone besides just Wisconsin. To my mind, there's just no way to deny that the WCHA has been dominant as a conference this year, and the way that the RPI is set up absolutely screws them for beating up on each other. (Something we usually complain about in the ECAC.)
Oh, and way to go Big Green!
Re: PWR Rank vs. KRACH Rank
Posted by: Robb (---.169.137.235.ts46v-07.otnc1.ftwrth.tx.charter.co)
Date: February 12, 2005 06:11PM
Very interesting way to look at it - nice!
I pretty much don't want to face ANY of the likely WCHA teams this year. The way things are shaking out, it doesn't look like there are going to be any low-seeded WCHA teams (like the Mankato patsies of '03) except for UND. I don't care what their ranking is - 7 titles says you don't want to face them in a 1-and-done.
I pretty much don't want to face ANY of the likely WCHA teams this year. The way things are shaking out, it doesn't look like there are going to be any low-seeded WCHA teams (like the Mankato patsies of '03) except for UND. I don't care what their ranking is - 7 titles says you don't want to face them in a 1-and-done.
Re: PWR Rank vs. KRACH Rank
Posted by: Chris 02 (---.icsincorporated.com)
Date: February 12, 2005 06:38PM
You might want to adjust this to compensate for ties in the PWR.
Re: PWR Rank vs. KRACH Rank
Posted by: Scersk '97 (---.dsl.chcgil.ameritech.net)
Date: February 12, 2005 07:10PM
Yep, better make sure that my "simplistic" comparison is hyper-accurate. 
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 13, 2005 09:42AM
From a common sense standpoint PWR seems more accurate to me than KRACH. First, it's possible to understand why PWR comes up with the ranking it does. Even though you may grasp KRACH's method and agree with it's logic, the window into why any two teams relate the way they do is not obvious. This transparency, alone, is enough to give PWR a significant advantage.
Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.
Look at WI vs Cornell for example. KRACH flips them. The logic for WI being "better", per KRACH, is SOS being better accounted for in that it values WI SOS to be twice that of Cornell's. In KRACH's theoretical world WI does, or will do, better against all good teams, and in round robin play will do better against the same teams.
But Cornell does have a better record against good teams (the top 50% of the KRACH list is virtually the same as TUC). And Cornell does have a better record against common opponents.
Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.
Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.
Look at WI vs Cornell for example. KRACH flips them. The logic for WI being "better", per KRACH, is SOS being better accounted for in that it values WI SOS to be twice that of Cornell's. In KRACH's theoretical world WI does, or will do, better against all good teams, and in round robin play will do better against the same teams.
But Cornell does have a better record against good teams (the top 50% of the KRACH list is virtually the same as TUC). And Cornell does have a better record against common opponents.
Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.
Re: PWR Rank vs. KRACH Rank
Posted by: Trotsky (---.frdrmd.adelphia.net)
Date: February 13, 2005 10:02AM
[Q]Ken '70 Wrote:
Why model reality when it's there to see in the first place?[/q]
This is pithy and it captures the absurdity of a lot of abstruse analysis, but I don't think it applies here. PWR and RPI are not models of reality, like scientific theories. They are not explanatory nor, even when applied correctly, predictive. They are standings for teams which don't play each other in a balanced schedule.
Although transparency is appealing when following the horserace, that has no bearing on how well the ranking system measures past performance and relative difficulty of opposition.
The optimal solution will (I'm an optimist) be (1) sound mathematically, (2) intuitive, and (3) transparent. But until we get there, that is the prioritization.
Why model reality when it's there to see in the first place?[/q]
This is pithy and it captures the absurdity of a lot of abstruse analysis, but I don't think it applies here. PWR and RPI are not models of reality, like scientific theories. They are not explanatory nor, even when applied correctly, predictive. They are standings for teams which don't play each other in a balanced schedule.
Although transparency is appealing when following the horserace, that has no bearing on how well the ranking system measures past performance and relative difficulty of opposition.
The optimal solution will (I'm an optimist) be (1) sound mathematically, (2) intuitive, and (3) transparent. But until we get there, that is the prioritization.
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 13, 2005 11:12AM
[Q]Trotsky Wrote:
The optimal solution will (I'm an optimist) be (1) sound mathematically, (2) intuitive, and (3) transparent. But until we get there, that is the prioritization.[/q]
Then we agree...COP, TUC (KRACH-ified if you want), and H2H are mathematically sound, intuitive and transparent. RPI is the bone of contention for some people, but not here. I yielded to the KRACH heads a long time ago, and it's OK with me if only one of four selection components doesn't meet 2 of your 3 your criteria since the rest do.
The optimal solution will (I'm an optimist) be (1) sound mathematically, (2) intuitive, and (3) transparent. But until we get there, that is the prioritization.[/q]
Then we agree...COP, TUC (KRACH-ified if you want), and H2H are mathematically sound, intuitive and transparent. RPI is the bone of contention for some people, but not here. I yielded to the KRACH heads a long time ago, and it's OK with me if only one of four selection components doesn't meet 2 of your 3 your criteria since the rest do.
Re: PWR Rank vs. KRACH Rank
Posted by: jtwcornell91 (---.no.no.cox.net)
Date: February 13, 2005 11:14AM
[Q]Ken '70 Wrote:
From a common sense standpoint PWR seems more accurate to me than KRACH. First, it's possible to understand why PWR comes up with the ranking it does. Even though you may grasp KRACH's method and agree with it's logic, the window into why any two teams relate the way they do is not obvious. This transparency, alone, is enough to give PWR a significant advantage.
[/Q]
PWR is a hodgepodge of criteria thrown together to handle coaches' and fans' grouses about being ranked below another team when anecdotal evidence makes them sound better. By the same logic, why should we bother using RPI at all when it's not "obvious" why one team ends up above another based on record and strength of schedule?
[Q]
Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.
[/Q]
Common opponents and head-to-head results are a more direct way to compare teams than games against the rest of the NCAA, but they deal with a much smaller sample. How you perform in a few games can override your performance for the whole rest of the season. And the problem with the TUC criterion is that it compares straight winning percentage, and it's possible to have very different schedule strengths within the subset of TUCs. Just look at the PWCs from 1999, when Quinnipiac had the 5th best record in the nation vs Teams Under Consideration: [slack.net]
[Q]
Look at WI vs Cornell for example. KRACH flips them. The logic for WI being "better", per KRACH, is SOS being better accounted for in that it values WI SOS to be twice that of Cornell's. In KRACH's theoretical world WI does, or will do, better against all good teams, and in round robin play will do better against the same teams.
[/Q]
Well, the actual logic is that the set of KRACH ratings constructed using all the games played by all the teams in the NCAA gives the best fit to actual results. The breakdown into SOS and winning ratio is just an illustrative tool.
[Q]But Cornell does have a better record against good teams (the top 50% of the KRACH list is virtually the same as TUC).[/Q]
Except that Cornell actually has a worse record against teams with a RRWP above .500 (which is the obvious way to define TUC using KRACH instead of RPI): [slack.net] Cornell is 8-4-3 and Wisconsin is 16-7-1.
And if we keep the current definition of a TUC but consider the strength of the TUCs actually played, Wisconsin also has the advantage in that criterion as well:
[slack.net]
If you want an anecdotal explanation for that, consider that of Cornell's 14 games against TUCs, only one was against a team in the top 10 of RPI or KRACH, while nine of Wisconsin's 18 games against TUCs were. So it's hard to say Cornell winning 61% of their TUC games is better than Wisconsin winning 58% of theirs.
[Q]And Cornell does have a better record against common opponents.[/Q]
Based on four games by Cornell (2-0 vs Yale and 0-1-1 vs Michigan State) and two games by Wisconsin (0-0-1 vs Yale and 0-1 vs Michigan State). Are those four games really as important as the teams' overall performance over the whole season?
[Q]Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.[/q]
Except the other criteria in the PWR are handled even worse than RPI, which at least has some consideration of schedule strength built into it, even if it's a broken one. If you really want to keep the hodgepodge of 1) overall performance 2) performance vs TUCs 3) performance vs common opponents and 4) each head-to-head win counts as much as any other criterion -- and I think #3 and #4 are giving too much weight to potentially just a few games -- then to do it right, you should 1) use KRACH instead of RPI and 2) define TUCs with KRACH instead of RPI and compare performance vs TUCs using the same strength-of-schedule method as KRACH. (In principle, you should also consider strength of schedule in the common opponents criteria, since one team could play more games against stronger common opponents, but let's leave that and H2H as-is for simplicity.) So program all those modifications into the DIY script (select c-HHWP to incorporate strength of schedule into any of the criteria that look at a subset of a team's games) and ...
[slack.net]
Cornell is still #7. In particular, we lose the comparison to Wisconsin because our KRACH is lower (although our RPI is higher) and our performance against TUCs is worse for the reasons explained above.
Similar recipes for how to improve PWR are spelled out in [slack.net] and in fact the system we proposed then looks like this with the DIY script: [slack.net]
From a common sense standpoint PWR seems more accurate to me than KRACH. First, it's possible to understand why PWR comes up with the ranking it does. Even though you may grasp KRACH's method and agree with it's logic, the window into why any two teams relate the way they do is not obvious. This transparency, alone, is enough to give PWR a significant advantage.
[/Q]
PWR is a hodgepodge of criteria thrown together to handle coaches' and fans' grouses about being ranked below another team when anecdotal evidence makes them sound better. By the same logic, why should we bother using RPI at all when it's not "obvious" why one team ends up above another based on record and strength of schedule?
[Q]
Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.
[/Q]
Common opponents and head-to-head results are a more direct way to compare teams than games against the rest of the NCAA, but they deal with a much smaller sample. How you perform in a few games can override your performance for the whole rest of the season. And the problem with the TUC criterion is that it compares straight winning percentage, and it's possible to have very different schedule strengths within the subset of TUCs. Just look at the PWCs from 1999, when Quinnipiac had the 5th best record in the nation vs Teams Under Consideration: [slack.net]
[Q]
Look at WI vs Cornell for example. KRACH flips them. The logic for WI being "better", per KRACH, is SOS being better accounted for in that it values WI SOS to be twice that of Cornell's. In KRACH's theoretical world WI does, or will do, better against all good teams, and in round robin play will do better against the same teams.
[/Q]
Well, the actual logic is that the set of KRACH ratings constructed using all the games played by all the teams in the NCAA gives the best fit to actual results. The breakdown into SOS and winning ratio is just an illustrative tool.
[Q]But Cornell does have a better record against good teams (the top 50% of the KRACH list is virtually the same as TUC).[/Q]
Except that Cornell actually has a worse record against teams with a RRWP above .500 (which is the obvious way to define TUC using KRACH instead of RPI): [slack.net] Cornell is 8-4-3 and Wisconsin is 16-7-1.
And if we keep the current definition of a TUC but consider the strength of the TUCs actually played, Wisconsin also has the advantage in that criterion as well:
[slack.net]
If you want an anecdotal explanation for that, consider that of Cornell's 14 games against TUCs, only one was against a team in the top 10 of RPI or KRACH, while nine of Wisconsin's 18 games against TUCs were. So it's hard to say Cornell winning 61% of their TUC games is better than Wisconsin winning 58% of theirs.
[Q]And Cornell does have a better record against common opponents.[/Q]
Based on four games by Cornell (2-0 vs Yale and 0-1-1 vs Michigan State) and two games by Wisconsin (0-0-1 vs Yale and 0-1 vs Michigan State). Are those four games really as important as the teams' overall performance over the whole season?
[Q]Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.[/q]
Except the other criteria in the PWR are handled even worse than RPI, which at least has some consideration of schedule strength built into it, even if it's a broken one. If you really want to keep the hodgepodge of 1) overall performance 2) performance vs TUCs 3) performance vs common opponents and 4) each head-to-head win counts as much as any other criterion -- and I think #3 and #4 are giving too much weight to potentially just a few games -- then to do it right, you should 1) use KRACH instead of RPI and 2) define TUCs with KRACH instead of RPI and compare performance vs TUCs using the same strength-of-schedule method as KRACH. (In principle, you should also consider strength of schedule in the common opponents criteria, since one team could play more games against stronger common opponents, but let's leave that and H2H as-is for simplicity.) So program all those modifications into the DIY script (select c-HHWP to incorporate strength of schedule into any of the criteria that look at a subset of a team's games) and ...
[slack.net]
Cornell is still #7. In particular, we lose the comparison to Wisconsin because our KRACH is lower (although our RPI is higher) and our performance against TUCs is worse for the reasons explained above.
Similar recipes for how to improve PWR are spelled out in [slack.net] and in fact the system we proposed then looks like this with the DIY script: [slack.net]
Edited 1 time(s). Last edit at 02/13/2005 11:16AM by jtwcornell91.
Pairwise comparisons vs single ranking system
Posted by: jtwcornell91 (---.no.no.cox.net)
Date: February 13, 2005 11:39AM
Another reason why I think the NCAA ought to drop the pairwise comparison system: the committee no longer applies it as it was intended. The pairwise comparison system tells you how team A relates to team B, so totalling up team A's comparisons with teams B, C, ... Z and team B's comparisons with teams A, C, ... Z is primarily useful for getting an overall feel for where the bubble is, but then you're supposed, as the system was originally conceived, to look at the comparisons among teams actually in the running for an at-large bid or a seed. The Joe Marsh committees did this, and the old You Are The Committee scripts let you walk through the process: [slack.net]
But lately, it's clear that the committee just looks at the ordering of teams by PWR and uses those 1-16 seeds to rank the teams. The clearest case was in 2003, when Ohio State and Harvard got 3 seeds and SCSU and Mankato got 4 seeds because they were 11, 12, 13, and 14 in the PWR, with 17, 16, 16, and 15 comparisons won, respectively. (Harvard wins the comparison with SCSU and has a higher RPI, so whichever one is the "tiebreaker" they get it.) But those overall PWR numbers include comparisons against teams that didn't make the field of 16. If you actually use the comparisons among tournament teams to seed the field, the number of comparisons won is 4, 4, 4, and 2. OSU, Harvard, and SCSU are in a rock-scissors-paper tie (OSU beats Harvard beats SCSU beats OSU), so we should go to the RPI to resolve it, and the order on that is Harvard, SCSU, OSU, which means OSU and Mankato should be 4 seeds and Harvard and SCSU should be 3 seeds. (In effect, St. Cloud got a 4 seed because they lost the pairwise comparison with Dartmouth, a team not even in the field.) So in fact that whole business with Cornell getting Mankato in the first round would have been avoided if the committee had used the PWCs as they were used in the past. (See [lists.maine.edu] for more.)
Having a pairwise comparison system is fine (as long as the criteria are sensible) but then it needs to get used as a pairwise comparison system. If you want an overall ranking (which is what the current committee seems to want), use a system that gives you an overall ranking.
But lately, it's clear that the committee just looks at the ordering of teams by PWR and uses those 1-16 seeds to rank the teams. The clearest case was in 2003, when Ohio State and Harvard got 3 seeds and SCSU and Mankato got 4 seeds because they were 11, 12, 13, and 14 in the PWR, with 17, 16, 16, and 15 comparisons won, respectively. (Harvard wins the comparison with SCSU and has a higher RPI, so whichever one is the "tiebreaker" they get it.) But those overall PWR numbers include comparisons against teams that didn't make the field of 16. If you actually use the comparisons among tournament teams to seed the field, the number of comparisons won is 4, 4, 4, and 2. OSU, Harvard, and SCSU are in a rock-scissors-paper tie (OSU beats Harvard beats SCSU beats OSU), so we should go to the RPI to resolve it, and the order on that is Harvard, SCSU, OSU, which means OSU and Mankato should be 4 seeds and Harvard and SCSU should be 3 seeds. (In effect, St. Cloud got a 4 seed because they lost the pairwise comparison with Dartmouth, a team not even in the field.) So in fact that whole business with Cornell getting Mankato in the first round would have been avoided if the committee had used the PWCs as they were used in the past. (See [lists.maine.edu] for more.)
Having a pairwise comparison system is fine (as long as the criteria are sensible) but then it needs to get used as a pairwise comparison system. If you want an overall ranking (which is what the current committee seems to want), use a system that gives you an overall ranking.
Re: PWR Rank vs. KRACH Rank
Posted by: ugarte (---.ny325.east.verizon.net)
Date: February 13, 2005 12:19PM
[Q]Ken '70 Wrote: This transparency, alone, is enough to give PWR a significant advantage.[/q]Win% is transparent. That doesn't make it particularly explanatory when the schedule is unbalanced. Hockey fans (and, for that matter, the selection committee) has shown themselves to be particularly tolerant of statistical analysis to measure relative strength. If KRACH were explained to the community at large I think the neutrality of the methodology would be considered transparent even if the audience (like, say, me) couldn't figure it out themselves with a supercomputer and a truckload of slide rules.
[q]Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.[/q]As John said, the sample sizes are so small that they are meaningless for anything except harrassing a fan of the other team in a bar. All I can think of is Bentley beat Quinnipiac ... who beat Denver, so Bentley should be #1! The real heavy lifting is in John's response.
[q]Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.[/q]Reality is in the eye of the beholder. If you think that one tie between Yale and Wisconsin is more representative of the relative strengths of those teams (and how we relate to both of them) than a recursive analysis of all games played by all teams, this isn't a debate. It is a clashing of realities.
[q]Instead of just simulation of how two teams relate because of schedules and their own results, PWR provides direct data in the form of COP. It creates a virtual "league of the best" in the form of TUC. Most potently, it includes H2H which is the most direct evidence of two team's relative strength.[/q]As John said, the sample sizes are so small that they are meaningless for anything except harrassing a fan of the other team in a bar. All I can think of is Bentley beat Quinnipiac ... who beat Denver, so Bentley should be #1! The real heavy lifting is in John's response.
[q]Why model reality when it's there to see in the first place? KRACH instead of RPI? Fine. KRACH instead of PWR? Silly.[/q]Reality is in the eye of the beholder. If you think that one tie between Yale and Wisconsin is more representative of the relative strengths of those teams (and how we relate to both of them) than a recursive analysis of all games played by all teams, this isn't a debate. It is a clashing of realities.
___________________________
quality tweets | bluesky (twitter 2) | ALAB Series podcast | Other podcasts and writing
quality tweets | bluesky (twitter 2) | ALAB Series podcast | Other podcasts and writing
Edited 1 time(s). Last edit at 02/13/2005 12:19PM by ugarte.
Re: Pairwise comparisons vs single ranking system
Posted by: The Rancor (---.2.156.11.lightlink.com)
Date: February 13, 2005 12:56PM
PWR and KRACH are both good for determining seeding for the tournament but cant real show intangible elements of a team. an example being after months of play and practice Cornell is a way better team than they were in October, by virtue of playing the season together and having a low occurrence of injury etc. also, they are clicking, winning and 'on a roll' mental momentum is a big factor. Could the February team kick Michigan State's ass? yes, i think so, even if the October team couldn't. also State is a slightly different team at this point in the season, I'd say worse now than then. but that's what 'polls' measure i guess, so that's why we need both.
Re: Pairwise comparisons vs single ranking system
Posted by: KeithK (---.dsl.snfc21.pacbell.net)
Date: February 13, 2005 09:18PM
[q]But lately, it's clear that the committee just looks at the ordering of teams by PWR and uses those 1-16 seeds to rank the teams.[/q]I think this is a case of USCHO doing too good of a job publlicixing the Pairwise. As far as I know, the coachesdidn't sit down and come up with the PWR table as it is presented on USCHO. They did comparisons of pairs of teams the way John described. Then USCHO presents them in tabular form and people get the mistaken opinion that it's a ranking system. The committee membership changes and the new guys start using it in the "improper" (that is, non-original) way because that is the general perception among fans and because that's more similar to the way most NCAA sports' selection process (RPI).
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 13, 2005 09:46PM
[Q]jtwcornell91 Wrote:
If you really want to keep the hodgepodge of 1) overall performance 2) performance vs TUCs 3) performance vs common opponents and 4) each head-to-head win counts as much as any other criterion -- you should 1) use KRACH instead of RPI and 2) define TUCs with KRACH instead of RPI and compare performance vs TUCs using the same strength-of-schedule method as KRACH. (In principle, you should also consider strength of schedule in the common opponents criteria, since one team could play more games against stronger common opponents, but let's leave that and H2H as-is for simplicity.) So program all those modifications into the DIY script (select c-HHWP to incorporate strength of schedule into any of the criteria that look at a subset of a team's games) and ...
Cornell is still #7. In particular, we lose the comparison to Wisconsin because our KRACH is lower (although our RPI is higher) and our performance against TUCs is worse for the reasons explained above.
[/q]
Great! We can combine advanced, recursive formulae with reality and get...an even better simulation of reality than we had before. Works for me.
If you really want to keep the hodgepodge of 1) overall performance 2) performance vs TUCs 3) performance vs common opponents and 4) each head-to-head win counts as much as any other criterion -- you should 1) use KRACH instead of RPI and 2) define TUCs with KRACH instead of RPI and compare performance vs TUCs using the same strength-of-schedule method as KRACH. (In principle, you should also consider strength of schedule in the common opponents criteria, since one team could play more games against stronger common opponents, but let's leave that and H2H as-is for simplicity.) So program all those modifications into the DIY script (select c-HHWP to incorporate strength of schedule into any of the criteria that look at a subset of a team's games) and ...
Cornell is still #7. In particular, we lose the comparison to Wisconsin because our KRACH is lower (although our RPI is higher) and our performance against TUCs is worse for the reasons explained above.
[/q]
Great! We can combine advanced, recursive formulae with reality and get...an even better simulation of reality than we had before. Works for me.
Re: PWR Rank vs. KRACH Rank
Posted by: Jim Hyla (---.twcny.res.rr.com)
Date: February 14, 2005 08:41PM
[Q]Ken '70 Wrote: Great! We can combine advanced, recursive formulae with reality and get...an even better simulation of reality than we had before. Works for me.
[/q]Define your reality, please.
[/q]Define your reality, please.
___________________________
"Cornell Fans Made the Timbers Tremble", Boston Globe, March/1970
Cornell lawyers stopped the candy throwing. Jan/2005
"Cornell Fans Made the Timbers Tremble", Boston Globe, March/1970
Cornell lawyers stopped the candy throwing. Jan/2005
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 14, 2005 09:24PM
[Q]Jim Hyla Wrote:
Ken '70 Wrote: Great! We can combine advanced, recursive formulae with reality and get...an even better simulation of reality than we had before. Works for me.
[/Q]
Define your reality, please.[/q]
MN beat WI 3 of 4 this year = reality
KRACH says WI is better than MN = unreality
Got it?
Re: PWR Rank vs. KRACH Rank
Posted by: DeltaOne81 (---.bos.east.verizon.net)
Date: February 14, 2005 10:36PM
That's far too small of a sample to matter as much as you are implying it should, Ken.
By that rational, we're equals with Dartmouth this year. Is that true? Or, Mich St is distinctly better than us, right?
What about that Minn is 0-2-1 against UAA, and Wisconsin of 4-0-0 against them? You have a 4 game sample directly between the two, that's a 7 game sample with only one intermediate. Bigger sample? Wisconsin 15-4-1 to Minn's 12-7-1 for Common Opponents.
In their conference? Wisconsin 15-6-1, Minn 13-10-1.
Obviously none of these pictures tell the whole story, but neither does the head to head. They're just one part of the picture. You can weight things however you want to personally, but to pull up the H2H record, and say that's the only thing that matters, is certainly not going to be a consensus opinion.
By that rational, we're equals with Dartmouth this year. Is that true? Or, Mich St is distinctly better than us, right?
What about that Minn is 0-2-1 against UAA, and Wisconsin of 4-0-0 against them? You have a 4 game sample directly between the two, that's a 7 game sample with only one intermediate. Bigger sample? Wisconsin 15-4-1 to Minn's 12-7-1 for Common Opponents.
In their conference? Wisconsin 15-6-1, Minn 13-10-1.
Obviously none of these pictures tell the whole story, but neither does the head to head. They're just one part of the picture. You can weight things however you want to personally, but to pull up the H2H record, and say that's the only thing that matters, is certainly not going to be a consensus opinion.
Re: PWR Rank vs. KRACH Rank
Posted by: jtwcornell91 (---.no.no.cox.net)
Date: February 14, 2005 11:37PM
[Q]Ken '70 Wrote:
MN beat WI 3 of 4 this year = reality
KRACH says WI is better than MN = unreality
Got it?
[/q]
RPI took 3 out of 4 points from Brown this year = reality
ECAC standings say Brown is better than RPI = unreality
MN beat WI 3 of 4 this year = reality
KRACH says WI is better than MN = unreality
Got it?
[/q]
RPI took 3 out of 4 points from Brown this year = reality
ECAC standings say Brown is better than RPI = unreality
Re: PWR Rank vs. KRACH Rank
Posted by: French Rage (---.Stanford.EDU)
Date: February 15, 2005 03:13AM
[Q]jtwcornell91 Wrote:
Common opponents and head-to-head results are a more direct way to compare teams than games against the rest of the NCAA, but they deal with a much smaller sample. How you perform in a few games can override your performance for the whole rest of the season. And the problem with the TUC criterion is that it compares straight winning percentage, and it's possible to have very different schedule strengths within the subset of TUCs. Just look at the PWCs from 1999, when Quinnipiac had the 5th best record in the nation vs Teams Under Consideration:
[/q]
Random question, but back then didnt they used to determine who was a TUC based on over .500 record instead of over .500 RPI?
Common opponents and head-to-head results are a more direct way to compare teams than games against the rest of the NCAA, but they deal with a much smaller sample. How you perform in a few games can override your performance for the whole rest of the season. And the problem with the TUC criterion is that it compares straight winning percentage, and it's possible to have very different schedule strengths within the subset of TUCs. Just look at the PWCs from 1999, when Quinnipiac had the 5th best record in the nation vs Teams Under Consideration:
[/q]
Random question, but back then didnt they used to determine who was a TUC based on over .500 record instead of over .500 RPI?
Re: PWR Rank vs. KRACH Rank
Posted by: jtwcornell91 (---.no.no.cox.net)
Date: February 15, 2005 06:20AM
[Q]French Rage Wrote:
jtwcornell91 Wrote:
Common opponents and head-to-head results are a more direct way to compare teams than games against the rest of the NCAA, but they deal with a much smaller sample. How you perform in a few games can override your performance for the whole rest of the season. And the problem with the TUC criterion is that it compares straight winning percentage, and it's possible to have very different schedule strengths within the subset of TUCs. Just look at the PWCs from 1999, when Quinnipiac had the 5th best record in the nation vs Teams Under Consideration:
[/Q]
Random question, but back then didnt they used to determine who was a TUC based on over .500 record instead of over .500 RPI?[/q]
Yes, so it was more extreme then, since it was easy for MAAC teams to become TUCs. Although it turns out that both of the other MAAC teams with winning records also had RPIs above .500. (Although only UConn and not Holy Cross would have with the 25-50-25 RPI formula.)
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 15, 2005 08:48PM
[Q]DeltaOne81 Wrote:
They're just one part of the picture. You can weight things however you want to personally, but to pull up the H2H record, and say that's the only thing that matters, is certainly not going to be a consensus opinion.[/q]
[aside]I don't think you're are paying attention. The substring you're responding to, started by me, proposes that PWR use KRACH instead of RPI for the SOS component and that KRACH-ifying TUC makes sense. I can only suppose, therefore, that your complaint above is that KRACH be used instead of PWR for ranking. If not, then be careful where you put your replies because you're taking the discussion out the context which clarifies it.[/aside]
Of course H2H is only part of the picture, that's my point exactly - it's part of the picture. Just like KRACH, or RPI, is part of the picture.
But KRACH by itself doesn't value H2H, it blends those games in with all the rest when comparing two teams - that's why it's not dealing properly with reality.
PWR gives extra weight, appropriately, to what happened on the ice between two teams in coming up with it's relative ranking of the two. And to what actually happened when those two teams played the same opponents. That's not relevant to a ranking system?
It seems that your (plural) real issue isn't just that KRACH's mechanics are better than RPI's or TUC's or COP's or H2H's or some combination thereof, but that the philosophy of an overall ranking by a method of individual comparisons is fundamentally wrong. Is that it?
P.S. The consensus opinion among the practitioners who count, the NCAA and its selection committee, is that H2H, COP etc are very important.
They're just one part of the picture. You can weight things however you want to personally, but to pull up the H2H record, and say that's the only thing that matters, is certainly not going to be a consensus opinion.[/q]
[aside]I don't think you're are paying attention. The substring you're responding to, started by me, proposes that PWR use KRACH instead of RPI for the SOS component and that KRACH-ifying TUC makes sense. I can only suppose, therefore, that your complaint above is that KRACH be used instead of PWR for ranking. If not, then be careful where you put your replies because you're taking the discussion out the context which clarifies it.[/aside]
Of course H2H is only part of the picture, that's my point exactly - it's part of the picture. Just like KRACH, or RPI, is part of the picture.
But KRACH by itself doesn't value H2H, it blends those games in with all the rest when comparing two teams - that's why it's not dealing properly with reality.
PWR gives extra weight, appropriately, to what happened on the ice between two teams in coming up with it's relative ranking of the two. And to what actually happened when those two teams played the same opponents. That's not relevant to a ranking system?
It seems that your (plural) real issue isn't just that KRACH's mechanics are better than RPI's or TUC's or COP's or H2H's or some combination thereof, but that the philosophy of an overall ranking by a method of individual comparisons is fundamentally wrong. Is that it?
P.S. The consensus opinion among the practitioners who count, the NCAA and its selection committee, is that H2H, COP etc are very important.
Re: PWR Rank vs. KRACH Rank
Posted by: DeltaOne81 (---.bos.east.verizon.net)
Date: February 15, 2005 08:52PM
I guess I did take it out of context. I have read this thread, but I can't necessarily go back through the whole thing each time.
I'm more on the side of a KRACH-ified PWR. Not sure if we should KRACH-ify components or just replace RPI with KRACH. But something along those lines. Really, I was responded to the fact that you seemed to be saying that Wisconsin is better than Minn, and right now there's no way I can agree with that. If that wasn't your point, then I'll try to read more context and maybe you try to give more context
I'm more on the side of a KRACH-ified PWR. Not sure if we should KRACH-ify components or just replace RPI with KRACH. But something along those lines. Really, I was responded to the fact that you seemed to be saying that Wisconsin is better than Minn, and right now there's no way I can agree with that. If that wasn't your point, then I'll try to read more context and maybe you try to give more context
Re: PWR Rank vs. KRACH Rank
Posted by: KeithK (---.external.lmco.com)
Date: February 15, 2005 09:09PM
I'm not convinced that head to head results are necessarily more meaningful than other games in determining which team of two teams is better. It's a question of sample sizes. One to four games (which is typically what you have) is small compared to the whole 30 game schedule (which is a small sample to begin with but it's what we've got). Just because Yale might get lucky and tie Wisconsin one night (or beat 'em) doesn't tell us anywhere near as much about their relative strengths as the rest of their schedules. Wisconsin is clearly better, and would be even if the Elis had somehow managed to beat the Badgers twice. Obviously in the Yale/Wisconsin case the weight most people would intuitively apply to the rest of the schedule makes the result of the overall comparison obvious. But in the Minnesota/Wisconsin comparison it's not quite so obvious.
I think we can all agree that the transitive rule doesn't apply to sports (A beats B, B beats C, therefore A is better than C). But the reason it doesn't apply is because the results of any one game don't tell you enough about the relative strengths of the two teams involved. This fact holds true when evaluating teams in a ranking system using H2H vs. entire schedules.
The only truly fair way to judge relative strength of a number of teams in a completely absolute fashion would be to play a round robin tournament between all the teams (identical schedules) with each team playing the others a very large number of times (sample size). Oh, and you'd have to play it in a short period of time so temporal effects are eliminated... Unfortunately I don't think that's really consistent with the whole student athlete (or human athlete) thing.
I think we can all agree that the transitive rule doesn't apply to sports (A beats B, B beats C, therefore A is better than C). But the reason it doesn't apply is because the results of any one game don't tell you enough about the relative strengths of the two teams involved. This fact holds true when evaluating teams in a ranking system using H2H vs. entire schedules.
The only truly fair way to judge relative strength of a number of teams in a completely absolute fashion would be to play a round robin tournament between all the teams (identical schedules) with each team playing the others a very large number of times (sample size). Oh, and you'd have to play it in a short period of time so temporal effects are eliminated... Unfortunately I don't think that's really consistent with the whole student athlete (or human athlete) thing.
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 15, 2005 09:13PM
[Q]jtwcornell91 Wrote:
Ken '70 Wrote:
MN beat WI 3 of 4 this year = reality
KRACH says WI is better than MN = unreality
Got it?
[/Q]
RPI took 3 out of 4 points from Brown this year = reality
ECAC standings say Brown is better than RPI = unreality[/q]
Hey, you're getting it, congratulations!
(In a world with just RPI and Brown, RPI is better. In world with all ECAC teams, Brown is doing better. In a world with more leagues and teams, deciding who is better is best accomplished by considering those facts plus how each did against the best teams each played and how they did against the common teams they played. Plus some formula that can't begin to yield results until dummy values are put in and which may need another dummy value to prevent a team like Cornell '70 from having a rating of infinity, which would make things a tad sticky. See, I'm getting it too!)
Re: PWR Rank vs. KRACH Rank
Posted by: RichH (---.stny.res.rr.com)
Date: February 15, 2005 10:42PM
Any thread where JTW is said to be "getting it" regarding this stuff is approaching a fine level of comedy, in my book. 
Re: PWR Rank vs. KRACH Rank
Posted by: Josh '99 (---.nyc.rr.com)
Date: February 16, 2005 03:22AM
Kinda seems like Ken is going in against a Sicilian when death is on the line. 
Re: PWR Rank vs. KRACH Rank
Posted by: Ken '70 (---.ne.client2.attbi.com)
Date: February 17, 2005 07:21PM
[Q]DeltaOne81 Wrote:
I guess I did take it out of context. I have read this thread, but I can't necessarily go back through the whole thing each time.
I'm more on the side of a KRACH-ified PWR. Not sure if we should KRACH-ify components or just replace RPI with KRACH. But something along those lines. Really, I was responded to the fact that you seemed to be saying that Wisconsin is better than Minn, and right now there's no way I can agree with that. If that wasn't your point, then I'll try to read more context and maybe you try to give more context [/q]
Fair enough. Thanks.
And I like a KRACH-ified PWR too.
I guess I did take it out of context. I have read this thread, but I can't necessarily go back through the whole thing each time.
I'm more on the side of a KRACH-ified PWR. Not sure if we should KRACH-ify components or just replace RPI with KRACH. But something along those lines. Really, I was responded to the fact that you seemed to be saying that Wisconsin is better than Minn, and right now there's no way I can agree with that. If that wasn't your point, then I'll try to read more context and maybe you try to give more context [/q]
Fair enough. Thanks.
And I like a KRACH-ified PWR too.
Sorry, only registered users may post in this forum.