Bracketology Starts

[BearLover]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

[RichH]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

[BearLover]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

I assume the above problems could be fixed pretty easily by just looking at, for example, how often the #1 team in the KRACH loses to the #2 team, how often the #1 loses to the #3, and so on. And then looking at how big an advantage home ice is, on average, and factoring that in.

[Beeeej]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

They're also not really designed to predict the probability of a particular outcome - they're designed to predict the probability of a particular outcome assuming everything between now and then pretty much goes the way it's gone up until now.

[KenP]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

They're also not really designed to predict the probability of a particular outcome - they're designed to predict the probability of a particular outcome assuming everything between now and then pretty much goes the way it's gone up until now.

One nice (and probably reliable) feature is the use of red vs green, which shows what the team can control. If we win out we are guaranteed no worse than #3.

[jfeath17]

I prefer this site. Shows where in the pairwise each team will finish based versus how many of their remaining games they win.

https://collegehockeyranked.com/forecast/pwrbywins/

[Jeff Hopkins '82]

I prefer this site. Shows where in the pairwise each team will finish based versus how many of their remaining games they win.

https://collegehockeyranked.com/forecast/pwrbywins/

Interesting.  And encouraging.  But it doesn't consider that we have 3-5 playoff games (hopefully 4) in addition to the 8 regular season games they show.

[KenP]

I prefer this site. Shows where in the pairwise each team will finish based versus how many of their remaining games they win.

https://collegehockeyranked.com/forecast/pwrbywins/

My only complaint with that site is that it ends with the regular season.  Not sure if that's true of the playoffstatus site.

[Trotsky]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

They're also not really designed to predict the probability of a particular outcome - they're designed to predict the probability of a particular outcome assuming everything between now and then pretty much goes the way it's gone up until now.

I don't agree at all with any of this line of criticism.  Roughly in order:

1. Just because intuitions disagree with odds doesn't mean the odds are wrong or the methodology is flawed.

2.  We operate with a host of perception biases and our monkey brains are notoriously terrible at assigning relative likelihoods and proportions because we are strongly influenced by anecdotal experience.

3. A model will of course "assume things keep on going as they have been" because that's the best guess of what will happen.  As long as the methodologically-correct degree of uncertainty (error) is built into the model, it's doing its job right.

Why is a 96% chance of making Lake Placid "obviously" absurd?  Sure, it seems high, but what it means is 19 times out of 20 team x with our profile would advance.  That's still once in 20 that it doesn't.  That may well be in accord with historical actuals.

The problem with models isn't modeling per se -- modeling works really well or Armstrong wouldn't have hit the moon.  What matters is taking care in choosing metrics and getting the probabilities right.  Statistics and probability theory have been working on each of those tasks respectively for a hundred years and as uncomfortable as it may be they now do a much, much better job of predicting than the eye test or common sense.

For a longer dissertation just read this entire site.

[adamw]

I think we had this discussion here last year. ... FWIW - as someone who wrote something similar for CHN (haven't published this year's yet - a couple more weeks until we do) ... I agree with everything Greg wrote.  Except I also agree with some of the criticisms, in that, I certainly believe it's possible to put some better metrics into the equation and come up with a better output.  Or - more precisely - I think the model could use some uncertainty fuzziness baked into it, so long as it's mathematically-based uncertainty.  And that is beyond my capabilities.

So if anyone wants to contribute any formulas, code or ideas to what we're doing on CHN - feel free. All ears.

[Beeeej]

I think we had this discussion here last year. ... FWIW - as someone who wrote something similar for CHN (haven't published this year's yet - a couple more weeks until we do) ... I agree with everything Greg wrote.  Except I also agree with some of the criticisms, in that, I certainly believe it's possible to put some better metrics into the equation and come up with a better output.  Or - more precisely - I think the model could use some uncertainty fuzziness baked into it, so long as it's mathematically-based uncertainty.  And that is beyond my capabilities.

So if anyone wants to contribute any formulas, code or ideas to what we're doing on CHN - feel free. All ears.

But then I wouldn't be able to criticize it. ::doh::

[KGR11]

We are now at a 90% probability of getting a 1-seed!

http://www.playoffstatus.com/ncaahockey/ncaahockeytournseedprob.html

I'm pretty sure this playoffstatus predictor is heavily flawed. It does not appear to sufficiently account for the inherent variance of a hockey game and considers a good team losing to a lesser team an extremely unlikely event. For example, it gives Cornell a 96% chance of making Lake Placid, which is obviously absurd. It also gives Cornell a 71% chance of getting the 1-seed in the ECAC and Clarkson a 27% chance, despite only one point separating them with eight games remaining and the two teams playing each other one more time. (Plus the game is at Clarkson and Clarkson crushed us the last time we played, which I assume the model doesn't take into account.)

Can't agree more. These "odds" sites that calculate and blend data down for any league you can think of really don't have any insight or context baked in.

They're also not really designed to predict the probability of a particular outcome - they're designed to predict the probability of a particular outcome assuming everything between now and then pretty much goes the way it's gone up until now.

I don't agree at all with any of this line of criticism.  Roughly in order:

1. Just because intuitions disagree with odds doesn't mean the odds are wrong or the methodology is flawed.

2.  We operate with a host of perception biases and our monkey brains are notoriously terrible at assigning relative likelihoods and proportions because we are strongly influenced by anecdotal experience.

3. A model will of course "assume things keep on going as they have been" because that's the best guess of what will happen.  As long as the methodologically-correct degree of uncertainty (error) is built into the model, it's doing its job right.

Why is a 96% chance of making Lake Placid "obviously" absurd?  Sure, it seems high, but what it means is 19 times out of 20 team x with our profile would advance.  That's still once in 20 that it doesn't.  That may well be in accord with historical actuals.

The problem with models isn't modeling per se -- modeling works really well or Armstrong wouldn't have hit the moon.  What matters is taking care in choosing metrics and getting the probabilities right.  Statistics and probability theory have been working on each of those tasks respectively for a hundred years and as uncomfortable as it may be they now do a much, much better job of predicting than the eye test or common sense.

For a longer dissertation just read this entire site.

More like 24 out of 25 ::bolt::
My big problem with playoffstatus is that their methodology isn't clear. How can I agree or disagree without knowing how they did it?

IIRC, CHN does a Monte Carlo analysis based on KRACH. I'm on board with that.

[adamw]

My big problem with playoffstatus is that their methodology isn't clear. How can I agree or disagree without knowing how they did it?

IIRC, CHN does a Monte Carlo analysis based on KRACH. I'm on board with that.

That is correct. I take the two KRACH values, and just use a random number generator to get the winner for that game, and every game. Then run it around 50,000 times, or as much as possible overnight.  But I do agree that KRACH might exaggerate things at the margins, given the relatively small sample sizes of past results we're dealing with.

[Trotsky]

I love his little trend thingy.

[Dafatone]

I love his little trend thingy.

Poor dead Alaska Anchorage.