2018 ECAC Permutations

[Jeff Hopkins '82]

Can we please create a separate thread for people to argue about mathematical models and statistics?

[BearLover]

My purpose in posting the percentages was to add new context to the discussion of KRACH's AQ results. BearLover doesn't buy the probability attributed to Cornell to win the AQ and I wanted to outline how it compares to other bye teams' AQ probabilities. I don't think this changes anyone's opinion, but it's an interesting metric to show how KRACH judges the top 4 ECAC teams.
Of course, part of the reason Cornell's probability is so high is that they face the easiest path as far as ECAC standings go. They'd be less likely to win if the tournament didn't reseed (I believe KRACH and intuition agree on this).

Okay, that's fair, and you make a good point about the last part. Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y. If one of the top four gets upset in the quarters, though, we would have an easier path.

In our six games this year against Union, Clarkson, and Harvard, we have a -1 goal differential (discounting Angello's empty-netter on the road vs. Harvard), and a -20 shot differential. We went 3-2-1 in those games, and each of those three teams have good goalies. So it's hard to figure we are more than a slight favorite against any of them.

[Dafatone]

My purpose in posting the percentages was to add new context to the discussion of KRACH's AQ results. BearLover doesn't buy the probability attributed to Cornell to win the AQ and I wanted to outline how it compares to other bye teams' AQ probabilities. I don't think this changes anyone's opinion, but it's an interesting metric to show how KRACH judges the top 4 ECAC teams.
Of course, part of the reason Cornell's probability is so high is that they face the easiest path as far as ECAC standings go. They'd be less likely to win if the tournament didn't reseed (I believe KRACH and intuition agree on this).

Okay, that's fair, and you make a good point about the last part. Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y. If one of the top four gets upset in the quarters, though, we would have an easier path.

In our six games this year against Union, Clarkson, and Harvard, we have a -1 goal differential (discounting Angello's empty-netter on the road vs. Harvard), and a -20 shot differential. We went 3-2-1 in those games, and each of those three teams have good goalies. So it's hard to figure we are more than a slight favorite against any of them.

Or to look at it through a different lens, we're 3-1 vs Harvard and union with the only loss being by one goal on the road. We're 0-1-1 vs Clarkson with that loss being fairly lopsided, but Clarkson has fallen off a cliff.

That's not to guarantee anything, of course.

[andyw2100]

Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y.

That last part isn't correct. We get the lowest remaining seed. There could be just one upset--say St. Lawrence over Dartmouth or RPI over Colgate--in which case we would play St. Lawrence or RPI.

[BearLover]

Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y.

That last part isn't correct. We get the lowest remaining seed. There could be just one upset--say St. Lawrence over Dartmouth or RPI over Colgate--in which case we would play St. Lawrence or RPI.

::doh::

[adamw]

The beauty of it is, what anyone thinks of Cornell's chances vs. one team or another doesn't matter. KRACH is what it is.  And there is nothing better that perfectly captures PAST results.  There are many "flaws" if you will to the model when it comes to projecting odds of winning future games - but I don't think they're flaws. They're just incomplete.  All of the reasons stated are valid.  But, as someone said, you'd need to come to the issue with actual valid math, and a better algorithm, before getting all hot and bothered about it.  Until then, saying that you "feel" that 52% chance is "flawed" is just as "flawed" of an argument as anything else.

Polls are a shit-ton more flawed than this is - doesn't stop Jim from posting them ... I've given him grief about it in the past, but all in fun. Would never tell him to stop.

Feel free to point out things all you want. But until you have a better model, and are willing to program it, then jeebus h. criminy, let people discuss it. It does a pretty fair job of giving you a portrait of what could happen.  I think everyone here (unlike many other places) is smart enough to know to take it with some grain of salt.  But it's as good a guideline as you've got.

[andyw2100]

Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y.

That last part isn't correct. We get the lowest remaining seed. There could be just one upset--say St. Lawrence over Dartmouth or RPI over Colgate--in which case we would play St. Lawrence or RPI.

::doh::

I can't tell if the head-smack is for me, because I'm missing something, or if it's an acknowledgment of the fact that you weren't thinking completely clearly when you made the initial post.

[ugarte]

Though assuming there is no more than one upset in the first round, I don't think we have an easier path in the quarters than any of the other bye teams, since we get Q/Y.

That last part isn't correct. We get the lowest remaining seed. There could be just one upset--say St. Lawrence over Dartmouth or RPI over Colgate--in which case we would play St. Lawrence or RPI.

::doh::

I can't tell if the head-smack is for me, because I'm missing something, or if it's an acknowledgment of the fact that you weren't thinking completely clearly when you made the initial post.

Decide if you are right or not and give BearLover credit for the appropriate response.

[KGR11]

The beauty of it is, what anyone thinks of Cornell's chances vs. one team or another doesn't matter. KRACH is what it is.  And there is nothing better that perfectly captures PAST results.  There are many "flaws" if you will to the model when it comes to projecting odds of winning future games - but I don't think they're flaws. They're just incomplete.  All of the reasons stated are valid.  But, as someone said, you'd need to come to the issue with actual valid math, and a better algorithm, before getting all hot and bothered about it.  Until then, saying that you "feel" that 52% chance is "flawed" is just as "flawed" of an argument as anything else.

Polls are a shit-ton more flawed than this is - doesn't stop Jim from posting them ... I've given him grief about it in the past, but all in fun. Would never tell him to stop.

Feel free to point out things all you want. But until you have a better model, and are willing to program it, then jeebus h. criminy, let people discuss it. It does a pretty fair job of giving you a portrait of what could happen.  I think everyone here (unlike many other places) is smart enough to know to take it with some grain of salt.  But it's as good a guideline as you've got.

Agreed. KRACH does an awesome job of ranking teams based on games played. For forecasting games, I think jfeath17's work could take it to the next level. In her logistic regression, the closer the KRACH winning percentage is to 100, the greater the difference between the KRACH winning percentage and the outcome winning percentage. I think this makes a lot of sense: if a team has a perfect record, they can still lose future games (example: 2007 Patriots and 2015 Kentucky basketball), so a team with a nearly perfect record should also have a lower probability of winning future games.

I think the biggest challenge for jfeath17 is that there's only 2 years of data. I think the next step is gather data for a couple of years and see how stable that logistic regression is year-to-year. jfeath17, I'd be interested to see a more detailed procedure that you used. That way, if you ended up stepping back, someone else could try this.

[Swampy]

The beauty of it is, what anyone thinks of Cornell's chances vs. one team or another doesn't matter. KRACH is what it is.  And there is nothing better that perfectly captures PAST results.  There are many "flaws" if you will to the model when it comes to projecting odds of winning future games - but I don't think they're flaws. They're just incomplete.  All of the reasons stated are valid.  But, as someone said, you'd need to come to the issue with actual valid math, and a better algorithm, before getting all hot and bothered about it.  Until then, saying that you "feel" that 52% chance is "flawed" is just as "flawed" of an argument as anything else.

Polls are a shit-ton more flawed than this is - doesn't stop Jim from posting them ... I've given him grief about it in the past, but all in fun. Would never tell him to stop.

Feel free to point out things all you want. But until you have a better model, and are willing to program it, then jeebus h. criminy, let people discuss it. It does a pretty fair job of giving you a portrait of what could happen.  I think everyone here (unlike many other places) is smart enough to know to take it with some grain of salt.  But it's as good a guideline as you've got.

Agreed. KRACH does an awesome job of ranking teams based on games played. For forecasting games, I think jfeath17's work could take it to the next level. In her logistic regression, the closer the KRACH winning percentage is to 100, the greater the difference between the KRACH winning percentage and the outcome winning percentage. I think this makes a lot of sense: if a team has a perfect record, they can still lose future games (example: 2007 Patriots and 2015 Kentucky basketball), so a team with a nearly perfect record should also have a lower probability of winning future games.

I think the biggest challenge for jfeath17 is that there's only 2 years of data. I think the next step is gather data for a couple of years and see how stable that logistic regression is year-to-year. jfeath17, I'd be interested to see a more detailed procedure that you used. That way, if you ended up stepping back, someone else could try this.

Two things are going on here. One is regression toward the mean, which is a valid observation because certain statistics necessarily behave this way. The other is the idea that if something happens once the probability of it happening again is lower.This is a fallacy.

[adamw]

Two things are going on here. One is regression toward the mean, which is a valid observation because certain statistics necessarily behave this way. The other is the idea that if something happens once the probability of it happening again is lower.This is a fallacy.

Isn't the problem with regression towards the mean, knowing what the mean is? It's not the same for every team.

Just another call by the way - begging for any all of you who have ideas, to come with me and work on an enhanced model for next year. I'd be more than happy to publish it.  Preferably more than one of you, so you can peer review each other

[Swampy]

Two things are going on here. One is regression toward the mean, which is a valid observation because certain statistics necessarily behave this way. The other is the idea that if something happens once the probability of it happening again is lower.This is a fallacy.

Isn't the problem with regression towards the mean, knowing what the mean is? It's not the same for every team.

Just another call by the way - begging for any all of you who have ideas, to come with me and work on an enhanced model for next year. I'd be more than happy to publish it.  Preferably more than one of you, so you can peer review each other

Well, probability and statistics has different levels of reality. The most obvious is observed empirical data. But there's also an assumption of an underlying, unobserved-but-real process that has certain probabilistic outcomes. But this applies to individual teams as well as to all teams in combination. Unless an individual team's true mean is a perfect season, which implies its probability of winning every individual game = 1.0, then since individual teams will regress towards their own means, an undefeated team will regress towards its own mean, which is < 1.0.

I don't like to explain this kind of stuff by saying things like, "assume Team X were to replay the season over 10,000 times" because it misrepresents what's actually going on with the math, and it concretizes what's actually an abstract, mathematical conceptualization. But let's do this for now.

Assume the actual probability distribution of Cornell's 1970 team going undefeated has an expected value of 0.95. In other words, if the team could replay the season an infinite number of times, 95% of the time it would go undefeated.

Since the quality of opponents varies each game, the probabilities of winning the individual games vary too. But for any given game there are two probabilities of interest. If Cornell is playing Harvard, for example, there's the probability Cornell will beat Harvard. If these given Cornell and Harvard teams were to play each other an infinite number of times, there's a certain underlying probability that Cornell would beat Harvard, but unless the distribution function of that probability has zero variance, Harvard has a non-zero probability of winning sometimes. (See my earlier post on variance.) The mean of the distribution function, its expected value, is the expected percentage of the time that Cornell would win. Suppose it really is 0.9, but after the first 10,000 games Cornell hasn't lost yet. Then, statistical theory says the tendency going forward would be for Cornell to lose because the mean really 0.9 and not 1.0, and one can prove mathematically that outcomes of probabilistic processes regress towards the mean.

The other probability is the probability of winning x games out of N games played. If x=N, then it's the probability of being undefeated at game N. The same logic applies. Knowing the "true" probabilities of winning an individual games against given opponents, we have probability distributions of winning Game 1, Game 2, etc. From these, we can construct a new variable, the probability of winning x games out of N. Again, unless this probability is 100%, then regression towards the mean implies that there's a higher likelihood an undefeated team will lose rather than win. This is because the expected value of the number of wins as of Game N, i.e. the mean, is < 1 but the number of wins up to that point = 1.

[BearLover]

But, as someone said, you'd need to come to the issue with actual valid math, and a better algorithm, before getting all hot and bothered about it.  Until then, saying that you "feel" that 52% chance is "flawed" is just as "flawed" of an argument as anything else.

I think this discussion is getting old too, but since some people keep saying those criticizing the model are doing so based on "feel," I just want to say that we really aren't. (a) jfeath17 already showed KRACH overstates the chances of higher-ranked teams winning an individual game. (b) When combining several artificially inflated individual probabilities together (Cornell's chances of winning the quarters, semis, and finals) to form one joint probability (Cornell winning the ECAC tournament), you end up with a very, very overly inflated likelihood (the 55% chance of Cornell winning the ECAC). (c) There are no betting odds for any NHL game that come close to the odds this model is assigning many games every weekend.

There may be confidence questions with jfeath17's model (ideally, we'd want more than 2 years of data), but it answers the question that's been raised: it takes the KRACH reported winning percentage and turns into the winning percentage that actually happened.

Right, changing the KRACH-inferred winning percentage to empirically based winning percentages would fix this problem with the model.

To make the model even more accurate would require throwing out KRACH or any ranking system that looks at only wins and losses, and instead measuring a team by goal differential, or better yet, shot differential (and adjusting for strength of schedule), but that's beyond the scope of my specific gripe with the model. (This isn't to say that ranking teams for tournament seeding/qualification purposes should look at anything other than wins/losses--KRACH is still the best at that.)

I can't tell if the head-smack is for me, because I'm missing something, or if it's an acknowledgment of the fact that you weren't thinking completely clearly when you made the initial post.

Me being dumb.

[adamw]

Right, changing the KRACH-inferred winning percentage to empirically based winning percentages would fix this problem with the model.

Is KRACH not empirically based?

To make the model even more accurate would require throwing out KRACH or any ranking system that looks at only wins and losses, and instead measuring a team by goal differential, or better yet, shot differential (and adjusting for strength of schedule), but that's beyond the scope of my specific gripe with the model. (This isn't to say that ranking teams for tournament seeding/qualification purposes should look at anything other than wins/losses--KRACH is still the best at that.)

It is not certain that looking at things beyond wins and losses is any better. Goal differential has major flaws, and might not mean much. Shot differential has its own issues, but could be a decent factor. Honestly, I'm not all that interested in things like goal and shot differential.

[Swampy]

I wonder, is there anything like KRACH in any competitive team sport that anyone here believes does a good job predicting outcomes of individual games?