Bracketology for 2020 NCAAs

[RichH]

Here's an example of the issue with using KRACH to predict final pairwise:

Cornell (KRACH Rating: 526) is playing St Lawrence (KRACH Rating: 11) in a few weeks. My understanding is that the ratings can be used to come up with a pseudo-record between the two teams (Cornell with 526 wins, St Lawrence with 11). The Monte Carlo simulation uses this record to determine how often Cornell wins. In this case, the model predicts they win 98% of the time.

jfeath's regression analysis from 2 years ago shows that a team with a KRACH winning percentage of 100% theoretically wins about 83.4% of the time, 14.5% lower than what KRACH states for the Cornell-St Lawrence game.

I think it makes sense for 83.4% to be an upper bound on winning percentage. Any goalie can have an incredible/incredibly bad day. Also, the fact that there are ties in hockey means that the winning percentage should be more weighted to 50% than a sport where you can't have ties.

Ideally, jfeath's regression analysis would be an in-between step in the Pairwise probability matrix to convert the KRACH winning percentages (which show what happened to date) to predictive winning percentages.

1st - this kind of disparity is extreme. St. Lawrence is historically lousy right now. I'd venture to say, Cornell probably would win 98% of the time.

2nd - ties are taken into consideration with a hard-coded 19% chance - which is already acting as a smoothing mechanism. It's probably not accurate to hard-code that value for every matchup - but it does act as a "smoother," so to speak. This lowers Cornell's chance of a win to closer to the 83% you're referring to. Cornell is winning 98% of the 81% percent of non-ties. So, 80% - plus 9.5%'s worth of win value (i.e. points) via the tie.

Hard-coding ties like that is a huge deal. You effectively changed the maximum winning percentage from 100% to 90%. That makes the simulation framework way better than I thought, since your max winning percentage is closer to jfeath's maximum winning percentage of 83%. The next time you publish a primer for the probability matrix, it might be worth including this.

I pointed to an extreme-disparity game because those are the games where KRACH overestimates the favorite team's winning percentage (per jfeath's analysis). That analysis shows that KRACH lines up pretty well when the favorite has a projected winning percentage of 70% or less. Somewhat of a moot point given the 19% tie input.

I'm tempted to propose a 50-1 bet for the Cornell-SLU game. Best case scenario, Cornell wins and I lose $2. Worst case scenario, Cornell loses and I win $100. Just seems a bit sacrilege.

Betting to profit off our misery is the plot of "The Big Short," so now I'm glad Topher is back tonight.

[upprdeck]

a couple interesting games this weekend.. Minn is now up to 16 and winning the B10 so that drops lowell out for now..

 Minn-PSU this weekend..  

PSU sweeps and that knocks Minn probably out of PWR contentio and takes them out of the B10 lead.  PSU then done for the regular season

Minn Sweeps  how far does PSU fall with only the playoffs left can they fall out of the top 15 with a bad playoff run.  

minn then plays Mich, and with the 3 pt win rules and almost anyone can still win that B10.

[osorojo]

Ice hockey is exclusively played on ice. Bracketology can be [and is] played on cell phones.

[Jeff Hopkins '82]

Ice hockey is exclusively played on ice. Bracketology can be [and is] played on cell phones.

And trolling is played on computers and cell phones.

[jtwcornell91]

Here's an example of the issue with using KRACH to predict final pairwise:

Cornell (KRACH Rating: 526) is playing St Lawrence (KRACH Rating: 11) in a few weeks. My understanding is that the ratings can be used to come up with a pseudo-record between the two teams (Cornell with 526 wins, St Lawrence with 11). The Monte Carlo simulation uses this record to determine how often Cornell wins. In this case, the model predicts they win 98% of the time.

jfeath's regression analysis from 2 years ago shows that a team with a KRACH winning percentage of 100% theoretically wins about 83.4% of the time, 14.5% lower than what KRACH states for the Cornell-St Lawrence game.

I think it makes sense for 83.4% to be an upper bound on winning percentage. Any goalie can have an incredible/incredibly bad day. Also, the fact that there are ties in hockey means that the winning percentage should be more weighted to 50% than a sport where you can't have ties.

Ideally, jfeath's regression analysis would be an in-between step in the Pairwise probability matrix to convert the KRACH winning percentages (which show what happened to date) to predictive winning percentages.

Even if the Bradley-Terry model is "correct" (whatever that means) there are two potential problems with using KRACH to predict the outcome of a mismatch:

One, KRACH is a maximum-likelihood estimate of a team's Bradley-Terry strength, whereas any estimate of the Bradley-Terry parameters based on a finite amount of data has some uncertainty in it.  Ordinarily that's not such a big deal for assigning probabilities to the outcome of one game: the ratio of Cornell's strength to Clarkson's might be higher or lower than our best guess, but that means we might have over- or under-estimated it, and so the uncertainty probably washes out.  But when the best guess is something like 50-to-1, that uncertainty can make a big difference in a more careful estimate of the probabilities.  As an oversimplified version, suppose the "correct" odds might be 100-to-1 or 25-to-1, but we don't know which.  Then the probability of an upset would be the average of 1.0% and 3.8%, which is 2.4% or about 40-to-1 against, not 50-to-1.  I.e., the uncertainty naturally biases our expectation of the true probability away from the extremes, because having maybe somewhat overestimated the magnitude of the upset is a bigger effect than having maybe somewhat underestimated it.  This is the issue we addressed in this paper, with a specific example discussed on this forum of the Cornell-Quinnipiac quarterfinal series from a few years back: http://dx.doi.org/10.13164/ma.2019.09 http://arxiv.org/abs/2001.04226

Two, the maximum-likelihood analysis doesn't take into account any prior expectations about the possible discrepancies in teams' strengths, which means it's equivalent to making your prior information completely noninformative.  This is a well-known effect which leads to undefeated teams having infinite KRACH ratings, and it's why Ken Butler put the "fictitious games" into KRACH for a while (the maximum likelihood estimates with fictitious games turn out to be the maximum a posteriori estimates with a particular prior distribution).  But this is almost always a pretty small effect by this point in the season, so we don't generally worry about it.  (BTW, the basic problem is older than hockey, since LaPlace was working on it circa 1800.  What's your best guess probability that an event will happen, given that it's never happened in some number of chances?  If you use the fraction of times you've already seen it as an estimate, you get zero, but you probably don't want to say it's literally impossible.  The Bayes-Laplace rule of succession is basically what you get if you at two extra "fictitious trials", one where it occurred and one where it didn't.)

Ties are a huge pain in the ass, and complicate everything, so it's often easier to pretend they don't exist (or rather that past ties are half wins and half losses and future ties are something we don't talk about), especially since they become impossible once the playoffs start.

[marty]

Here's an example of the issue with using KRACH to predict final pairwise:

Cornell (KRACH Rating: 526) is playing St Lawrence (KRACH Rating: 11) in a few weeks. My understanding is that the ratings can be used to come up with a pseudo-record between the two teams (Cornell with 526 wins, St Lawrence with 11). The Monte Carlo simulation uses this record to determine how often Cornell wins. In this case, the model predicts they win 98% of the time.

jfeath's regression analysis from 2 years ago shows that a team with a KRACH winning percentage of 100% theoretically wins about 83.4% of the time, 14.5% lower than what KRACH states for the Cornell-St Lawrence game.

I think it makes sense for 83.4% to be an upper bound on winning percentage. Any goalie can have an incredible/incredibly bad day. Also, the fact that there are ties in hockey means that the winning percentage should be more weighted to 50% than a sport where you can't have ties.

Ideally, jfeath's regression analysis would be an in-between step in the Pairwise probability matrix to convert the KRACH winning percentages (which show what happened to date) to predictive winning percentages.

Even if the Bradley-Terry model is "correct" (whatever that means) there are two potential problems with using KRACH to predict the outcome of a mismatch:

One, KRACH is a maximum-likelihood estimate of a team's Bradley-Terry strength, whereas any estimate of the Bradley-Terry parameters based on a finite amount of data has some uncertainty in it.  Ordinarily that's not such a big deal for assigning probabilities to the outcome of one game: the ratio of Cornell's strength to Clarkson's might be higher or lower than our best guess, but that means we might have over- or under-estimated it, and so the uncertainty probably washes out.  But when the best guess is something like 50-to-1, that uncertainty can make a big difference in a more careful estimate of the probabilities.  As an oversimplified version, suppose the "correct" odds might be 100-to-1 or 25-to-1, but we don't know which.  Then the probability of an upset would be the average of 1.0% and 3.8%, which is 2.4% or about 40-to-1 against, not 50-to-1.  I.e., the uncertainty naturally biases our expectation of the true probability away from the extremes, because having maybe somewhat overestimated the magnitude of the upset is a bigger effect than having maybe somewhat underestimated it.  This is the issue we addressed in this paper, with a specific example discussed on this forum of the Cornell-Quinnipiac quarterfinal series from a few years back: http://dx.doi.org/10.13164/ma.2019.09 http://arxiv.org/abs/2001.04226

Two, the maximum-likelihood analysis doesn't take into account any prior expectations about the possible discrepancies in teams' strengths, which means it's equivalent to making your prior information completely noninformative.  This is a well-known effect which leads to undefeated teams having infinite KRACH ratings, and it's why Ken Butler put the "fictitious games" into KRACH for a while (the maximum likelihood estimates with fictitious games turn out to be the maximum a posteriori estimates with a particular prior distribution).  But this is almost always a pretty small effect by this point in the season, so we don't generally worry about it.  (BTW, the basic problem is older than hockey, since LaPlace was working on it circa 1800.  What's your best guess probability that an event will happen, given that it's never happened in some number of chances?  If you use the fraction of times you've already seen it as an estimate, you get zero, but you probably don't want to say it's literally impossible.  The Bayes-Laplace rule of succession is basically what you get if you at two extra "fictitious trials", one where it occurred and one where it didn't.)

Ties are a huge pain in the ass, and complicate everything, so it's often easier to pretend they don't exist (or rather that past ties are half wins and half losses and future ties are something we don't talk about), especially since they become impossible once the playoffs start.

I'm thinking LaPlace spent more time on this problem than he did arguing with Amoureux des Ours.::bolt::

[abmarks]

Question for either everyone or just those with admin powers:

Can we have a dedicated thread for all things KRACH, PWR, probablities etc?  And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

Less thread drift in this would be great, plus the historical info on the subjects would still be in the same threads for reference.

[Trotsky]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

[nshapiro]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

Maybe we need a thread to discuss thread drift.

[Trotsky]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

Maybe we need a thread to discuss thread drift.

No, we should just discuss that in every thread...

[marty]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

Maybe we need a thread to discuss thread drift.

No, we should just discuss that in every thread...

I think what's missing is the statistical analysis of thread drift.  One should be able to predict the direction and verbosity of the drift based on the number of times each registered user accesses eLynah.  JTW might have some spare time to tackle this.

[Trotsky]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

Maybe we need a thread to discuss thread drift.

No, we should just discuss that in every thread...

I think what's missing is the statistical analysis of thread drift.  One should be able to predict the direction and verbosity of the drift based on the number of times each registered user accesses eLynah.  JTW might have some spare time to tackle this.

Only sissies use advanced metrics to predict thread drift.  Real men use the eye test.

[abmarks]

And if so, more. Importantly, can we move posts and or threads that drift into the theory and minutia debates over to there in the future as they crop up?

This is silly.  If you have a problem with drift ask somebody to move their own post.  Don't burden the mods with it.

Fair enough.  I'm used to mod-driven forums to assumed it was the same on here.

[Jim Hyla]

USCHO: Bracketology: Which bubble teams have a shot at playing for an NCAA hockey national championship?  

Jayson's

Worcester

1 North Dakota
7 Clarkson
9 Northeastern
16 AIC

Allentown

2 Minnesota State
8 Massachusetts
10 Penn State
15 Minnesota

Albany

3 Cornell
6 Boston College
11 Arizona State
14 Maine

Loveland

4 Minnesota Duluth
5 Denver
12 Bemidji State
13 Ohio State

Jim's

Albany

1 North Dakota
7 Clarkson
9 Northeastern
16 AIC

Allentown

2 Minnesota State
8 Massachusetts
10 Penn State
15 Minnesota

Worcester

3 Cornell
6 Boston College
11 Arizona State
14 Maine

Loveland

4 Minnesota Duluth
5 Denver
12 Bemidji State
13 Ohio State

[marty]

USCHO: Bracketology: Which bubble teams have a shot at playing for an NCAA hockey national championship?  

Jayson's

Worcester

1 North Dakota
7 Clarkson
9 Northeastern
16 AIC

Allentown

2 Minnesota State
8 Massachusetts
10 Penn State
15 Minnesota

Albany

3 Cornell
6 Boston College
11 Arizona State
14 Maine

Loveland

4 Minnesota Duluth
5 Denver
12 Bemidji State
13 Ohio State

Jim's

Albany

1 North Dakota
7 Clarkson
9 Northeastern
16 AIC

Allentown

2 Minnesota State
8 Massachusetts
10 Penn State
15 Minnesota

Worcester

3 Cornell
6 Boston College
11 Arizona State
14 Maine

Loveland

4 Minnesota Duluth
5 Denver
12 Bemidji State
13 Ohio State

Marty says "North Dakota earned the right to stay out west!" And "Why give Duluth an easier path to three-peat?"

This also allows for less travel for the teams in Allentown and Worcester.

Albany

3  Cornell[/u]
14 Maine

8  U Mass
7  Clarkson


Allentown

4   Minnesota Duluth
13 tOSU

9  Northeastern
10 Penn State


Worcester

2  Minn State Mankato
15 Mass Lowell

6 Boston College
11 Arizona State


Loveland

1  North Dakota
16 AIC

5 Denver
12 Bemidji State