Sunday, June 24th, 2018
Who is this season's most improved player?
Matt Buckles
Jeff Kubiak
Alex Rauter
Trevor Yates
 
 
 
Updates automatically
Twitter Link
CHN iOS App
 
NCAA
1967 1970

ECAC
1967 1968 1969 1970 1973 1980 1986 1996 1997 2003 2005 2010

IVY
1966 1967 1968 1969 1970 1971 1972 1973 1977 1978 1983 1984 1985 1996 1997 2002 2003 2004 2005 2012 2014

Cleary Spittoon
2002 2003 2005

Ned Harkness Cup
2003 2005 2008 2013
 
Brendon
Iles
Pokulok
Schafer
Syphilis

2018 ECAC Permutations

Posted by Give My Regards 
Page: Previous1 2 
Current Page: 2 of 2
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: Trotsky (---.dc.dc.cox.net)
Date: March 02, 2018 05:32PM

adamw
abmarks
it would be interesting to run a KRACH computation for last year's full NHL regular season, for example, and then see if the numbers pass people's gut checks or not.

This is honestly an unnecessary exercise. For past results, it's hard to improve on KRACH. The KRACH ratings, if you played the schedule that already happened, would come out to the actual results. That's the whole point of KRACH's existence.

How about this exercise. Number all the NC$$ games in chron order. Calculate KRACH from the odd number games. Now compare how the even numbered games turned out against KRACH "predictions."

I know, I know, I know that KRACH reviews a data set and is not designed to be predictive. But... can you do that for shits and giggles? I'm not even sure how we'd interpret the results. What constitutes a reliable or unreliable percentage of accuracy? I mean, hopefully it's over 50%. Hopefully it's better than just taking whoever has the better winning percentage excluding games against each other.

Another method: start on game 1 and just march through the list constantly recalculating KRACH and using that as the prediction against the next game. Or since obviously KRACH gets better as the season goes on, iterate through say the first 10% of games and only start predicting after that. Now get your accuracy score. That's truly abusing KRACH as predictive. :)
Edited 5 time(s). Last edit at 03/02/2018 05:38PM by Trotsky.
 
Re: 2018 ECAC Permutations
Posted by: BearLover (---.nycmny.ftas.verizon.net)
Date: March 03, 2018 01:04AM

adamw
Is KRACH not empirically based?
The future win probabilities inferred from KRACH aren't, because they're not verifiable by observation/experience.

adamw
It is not certain that looking at things beyond wins and losses is any better. Goal differential has major flaws, and might not mean much. Shot differential has its own issues, but could be a decent factor. Honestly, I'm not all that interested in things like goal and shot differential.
I think in the hockey analytics world it actually is pretty certain that looking at things beyond wins and losses is better.

I think it's helpful to think of it this way: only looking at game outcomes is a very small sample size. Goals, of which there are several in the average game, is a bigger sample. Shots is the biggest sample of all. And, in fact, shot differential, in serving as the best proxy we have for possession, does a tremendous job of measuring the strength of a team and thereby predicting the outcome of a hockey game.

This discussion often comes up on here when Cornell has a better record than its possession numbers would suggest and half of ELynah thinks the team will regress and the other half thinks they won't. Which necessarily leads to a discussion of whether possession is the be-all-end-all in college like it is in the pros. I won't rehash the arguments here, but shot differential is still so heavily correlated with wins in college that I highly doubt there exists a more predictive stat over the course of a regular season than shot differential.

abmarks
Well said. Some people are taking jfeath's work as gospel, when in fact all we got was a simple chart showing krach win% v actual for specific, arbitrary bands. Noone double-checked the work, either.
jfeath's work is just a very preliminary exercise that confirms what many of us suspected when we looked at some of the numbers these prediction models were pumping out. It also comports with NHL betting odds. NHL betting odds almost never give a team a less than a 1-in-3 chance of winning. Yet a few weeks ago the KRACH-based model was giving Cornell an 80% chance of beating Union and Harvard!
Edited 1 time(s). Last edit at 03/03/2018 01:07AM by BearLover.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: KGR11 (---.dyn.optonline.net)
Date: March 03, 2018 08:07AM

Trotsky
adamw
abmarks
it would be interesting to run a KRACH computation for last year's full NHL regular season, for example, and then see if the numbers pass people's gut checks or not.

This is honestly an unnecessary exercise. For past results, it's hard to improve on KRACH. The KRACH ratings, if you played the schedule that already happened, would come out to the actual results. That's the whole point of KRACH's existence.

How about this exercise. Number all the NC$$ games in chron order. Calculate KRACH from the odd number games. Now compare how the even numbered games turned out against KRACH "predictions."

I know, I know, I know that KRACH reviews a data set and is not designed to be predictive. But... can you do that for shits and giggles? I'm not even sure how we'd interpret the results. What constitutes a reliable or unreliable percentage of accuracy? I mean, hopefully it's over 50%. Hopefully it's better than just taking whoever has the better winning percentage excluding games against each other.

Another method: start on game 1 and just march through the list constantly recalculating KRACH and using that as the prediction against the next game. Or since obviously KRACH gets better as the season goes on, iterate through say the first 10% of games and only start predicting after that. Now get your accuracy score. That's truly abusing KRACH as predictive. :)

I think your second method is essentially what jfeath did, right?
 
Re: 2018 ECAC Permutations
Posted by: billhoward (---.nwrk.east.verizon.net)
Date: March 03, 2018 12:12PM

Sometimes I think it's pronounced krock.
 
Re: 2018 ECAC Permutations
Posted by: adamw (---.phlapa.fios.verizon.net)
Date: March 03, 2018 12:35PM

BearLover
adamw
Is KRACH not empirically based?
The future win probabilities inferred from KRACH aren't, because they're not verifiable by observation/experience.

What future probabilities of any kind are verifiable?

BearLover
adamw
It is not certain that looking at things beyond wins and losses is any better. Goal differential has major flaws, and might not mean much. Shot differential has its own issues, but could be a decent factor. Honestly, I'm not all that interested in things like goal and shot differential.
I think in the hockey analytics world it actually is pretty certain that looking at things beyond wins and losses is better.

There is no need to quote me articles about analytics. I deal with NHL analytics all day for my "real" job. The analytics community has also, finally, thank goodness, moved beyond its original rudimentary hypotheses about how hockey works. Shot differential as a proxy for possession was a nice tool in the toolbelt, but had/has a long way to go to create real understanding. There is shot quality, location data, rolling score effects, etc... finally being taken into consideration, and of course there is a lot in hockey that simply can't be measured yet. So while shot differential displayed some correlation to better predicting wins/losses than past wins and losses, it's really very rudimentary and there's plenty more to do.

However, you glossed over the fact that I said "goal differential" first. I don't know of any model that takes into account shot differential in ranking systems. Goal differential is another thing. There have been plenty of them that do. And that debate has gone on forever. My point was that goal differential has numerous flaws when it comes to hockey team ratings, which is why it's perfectly valid to ignore it when it comes to ratings systems, and probably predictive models. I also clearly said that shot differential could be a "decent factor" but has issues. So I'm not sure why the need to inform me that looking beyond wins and losses may be better. Well aware. No one ever said otherwise.

BearLover
I think it's helpful to think of it this way: only looking at game outcomes is a very small sample size. Goals, of which there are several in the average game, is a bigger sample. Shots is the biggest sample of all. And, in fact, shot differential, in serving as the best proxy we have for possession, does a tremendous job of measuring the strength of a team and thereby predicting the outcome of a hockey game.

Please stop with the Analytic-splaining ... Believe me, we all understand about sample sizes. Again, goal differential in hockey is flawed. The greater sample size there is not necessarily an improvement. And I would not call shot differential metrics doing a "tremendous job" ... It does a better job. Not a tremendous job. There are more factors. But sure, on the team level, it holds some weight. If it can be incorporated into predictive models, then great. But it's not a panacea.

BearLover
It also comports with NHL betting odds. NHL betting odds almost never give a team a less than a 1-in-3 chance of winning. Yet a few weeks ago the KRACH-based model was giving Cornell an 80% chance of beating Union and Harvard!

There's nothing more flawed than quoting betting odds, which bear no resemblance to anything except where money goes. I'm pretty sure the NHL KRACH figures I posted earlier demonstrate that the variance between NHL teams is far smaller than college teams. So comparing betting odds of NHL games to college possibilities is silly. Of course betting odds are never that wide on NHL games.

Again - we all get that there are better ways to do things. But I don't understand all the bellyaching about it. Many of your solutions have plenty of issues themselves. Come up with a model, and lay out the math, have it reviewed for problems, and I'll be more than happy to put it together. I haven't seen anyone be willing to do that yet.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: BearLover (---.nycmny.ftas.verizon.net)
Date: March 03, 2018 02:26PM


What future probabilities of any kind are verifiable?
What I mean to say is that the probabilities derived purely from KRACH are not back-tested against actual results, in the way 538's and a model like this one's are. The KRACH-based model instead takes what has happened in the past and extrapolates it into the future, and no one even checks to see how much it misses by.

adamw
There is no need to quote me articles about analytics. I deal with NHL analytics all day for my "real" job. The analytics community has also, finally, thank goodness, moved beyond its original rudimentary hypotheses about how hockey works. Shot differential as a proxy for possession was a nice tool in the toolbelt, but had/has a long way to go to create real understanding. There is shot quality, location data, rolling score effects, etc... finally being taken into consideration, and of course there is a lot in hockey that simply can't be measured yet. So while shot differential displayed some correlation to better predicting wins/losses than past wins and losses, it's really very rudimentary and there's plenty more to do.
I agree with all of this. But just because shots as a statistic is rudimentary doesn't mean it isn't the best tool we have right now.

adamw
However, you glossed over the fact that I said "goal differential" first. I don't know of any model that takes into account shot differential in ranking systems. Goal differential is another thing. There have been plenty of them that do. And that debate has gone on forever. My point was that goal differential has numerous flaws when it comes to hockey team ratings, which is why it's perfectly valid to ignore it when it comes to ratings systems, and probably predictive models. I also clearly said that shot differential could be a "decent factor" but has issues. So I'm not sure why the need to inform me that looking beyond wins and losses may be better. Well aware. No one ever said otherwise.
Again, you're conflating "is rudimentary/has its own sets of issues" with "is worse." Goals and shots aren't perfect, but they're better predictors than wins.

adamw
Please stop with the Analytic-splaining ...
Sorry about the analytic-splaining, but I can't recall an analytics study/article in the past six or so years I've been following this that concluded wins/losses is a better metric of future success than goals, and I don't know if I even recall an article/study that concluded goals were a better metric than shots. In fact, it seems every article/study leads with the assumption that Corsi/Fenwick is the best predictor we currently have, and goes from there. So you writing that you are not interested in a model that looks at goals/shots rather than wins suggested to me you aren't as familiar with current prediction models. I was wrong about your lack of familiarity, so you are welcome to post things that would back up your disdain for shot differential as a predictive stat relative to win% and goal-differential.

adamw
There's nothing more flawed than quoting betting odds, which bear no resemblance to anything except where money goes.
I posted betting odds because I wasn't aware of an actual NHL prediction model that gave probabilities for individual games. I've since found one, and it turns out I was wrong about the upper bounds of hockey probabilities: while the majority of its predictions are closer to 50% than those of KRACH, this model yields probabilities for certain games that are as high as 80%. The model seems to care less about a team's entire body of work and instead weights factors such as recent performance and starting goalie quality very highly--though it doesn't appear to release its entire methodology. The model claims to be about 60% accurate, which is about as good as it gets for NHL prediction models. On the other hand, the fact that this model is deriving these relatively lopsided probabilities not from total record but from all these other stats doesn't really help the case for a KRACH model, which looks only at past win% (and in fact there are a lot of probabilities from this model that show a matchup between teams with similar records as lopsided).

adamw
I'm pretty sure the NHL KRACH figures I posted earlier demonstrate that the variance between NHL teams is far smaller than college teams.
Yes, this is true. But is the gap between Cornell and Union/Harvard as big as the largest gap between any two NHL teams?
Edited 1 time(s). Last edit at 03/03/2018 02:27PM by BearLover.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jfeath17 (---.bstnma.fios.verizon.net)
Date: March 03, 2018 03:54PM

KGR11
Trotsky

Another method: start on game 1 and just march through the list constantly recalculating KRACH and using that as the prediction against the next game. Or since obviously KRACH gets better as the season goes on, iterate through say the first 10% of games and only start predicting after that. Now get your accuracy score. That's truly abusing KRACH as predictive. :)

I think your second method is essentially what jfeath did, right?

Yes, that is basically what I did except I stepped through week by week and started in January. I now have updated data which includes 4 seasons and steps through on a daily basis starting in January.

BearLover
adamw
Is KRACH not empirically based?
The future win probabilities inferred from KRACH aren't, because they're not verifiable by observation/experience.

What future probabilities of any kind are verifiable?

While you can't perfectly verify a prediction model, you can get an idea of its performance by separating the past data into training and testing sets. The fact that we are trying to predict probabilities and not simple classification does make it much more difficult to evaluate the performance. For classification problems the predictor is either right or wrong so it is easy to state a accuracy percentage. We however cannot directly observe the outcome probability of some matchup but only outcome of one trial of this matchup. This brings us to what I am attempting to do. By looking at the outcomes of many games with a similar krach predicted winning percentage we can come up with an estimate for the actual winning percentage of a team in this matchup.

My methodology for this was to use a gaussian weighted average of the games centered at varying krach probabilities. I calculated the winning percentage using these weights. I also used the weights to come up with average krach probability (this doesn't necessarily line up with the center of the gaussian particularly at the endpoints where all the games are to one side or the other). This is basically the logical extrapolation of the binning that was suggested earlier in the thread. The binning was actually the first analysis I did but the data didn't look good. I think the major improvement here is not that I am using the gaussian to come up with weights (that is probably overkill), but that I am using the weighted average of the krach probabilities rather than the center of the bin. What this trend line looks like can be changed significantly by changing the std dev of the gaussian (effectively changing the bin size). Basically the larger the bin the more underfit and the smaller the more overfit.


I also sought to measure the performance of KRACH in another way by looking at the R2 (Coefficient of Determination) The Wikipedia page is a pretty good explanation of this. The R^2 value can be looked at as the percentage of variance in the dependent variable (game outcomes) that can be explained by the independent variable (krach probability, etc..). These probabilities are all very low which makes sense since there is a lot of variability in the outcome of hockey games.

Independent Variable | R^2
--------------------------------------------------
Krach Probability | 0.023
Logistic Regression | 0.100
Linear Fit on Gaussian Average (0.1) | 0.112
(y=.749x+.126)


Another improvement I made was to include the inverse of each game (prob = 1-prob and swap wins and losses). This improves the fit around 0.5 since it is a little nonsensical if the matchup of two equal KRACH teams is not 0.5 in a model only dependent on KRACH.


One final point which I think has been established at this point, but I want to make sure we are all on the same page. Predictive models are going to have some subjectivity built into them. It is great that KRACH has no subjectivity and is a mathematically pure ranking for its goal of NCAA seeding since that needs to be "fair" and should be based on the actual outcomes of the games. However when creating a predictive model, we unfortunately do not have the luxury of a mathematically pure system. There are parameters and methods that must be chosen both when designing a predictor and measuring the performance. It is the designers goal to choose these such that the predictor is not over/underfit or have any bias's built in.

 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: adamw (---.phlapa.fios.verizon.net)
Date: March 03, 2018 07:06PM

BearLover
Again, you're conflating "is rudimentary/has its own sets of issues" with "is worse." Goals and shots aren't perfect, but they're better predictors than wins.

Actually, I think you're the only one conflating anything, because I never said "worse" - so I'm not sure where this is coming from.

This thread is very enjoyable to me, and I want to hear all of these things. And I'm pretty sure I've been clear that KRACH can be improved upon. And I agree with what you say, in general. What has rubbed me the wrong way with your posts is your unnecessary (to me) vehemence against KRACH, the twisting of what I've said, and the high-level of self-confidence in what you're saying. A little humility is helpful here because no one really knows.

BearLover
Sorry about the analytic-splaining, but I can't recall an analytics study/article in the past six or so years I've been following this that concluded wins/losses is a better metric of future success than goals, and I don't know if I even recall an article/study that concluded goals were a better metric than shots. In fact, it seems every article/study leads with the assumption that Corsi/Fenwick is the best predictor we currently have, and goes from there. So you writing that you are not interested in a model that looks at goals/shots rather than wins suggested to me you aren't as familiar with current prediction models. I was wrong about your lack of familiarity, so you are welcome to post things that would back up your disdain for shot differential as a predictive stat relative to win% and goal-differential.

The problem is you continuing to insist I said things that I never said. You are claiming that I am against shots/goals differential, when I never said anything of the sort. I just said it has its own issues. Particularly goal differential. Is a 6-2 win more indicative of quality than 6-4? Not necessarily in my opinion, and I think studies of such are flawed. Of course, there's empty-net goals to take into account, which is unique to hockey. Of course, those can be weeded out.

The older Analytics studies have not just been improved upon - in some cases, they have been downright contradicted. There is too much more involved in hockey than to be so confident in what these things are telling us so far. I've had issues with these studies from day one. And thankfully now they are being improved upon and we are finding out different things. Heck, Bill James spent most of his career telling everyone that clutch didn't exist, and now he spent the last year telling everyone that we should taken into account situational hitting when it comes to voting for the MVP Award.

So again, all I'm saying is, there is room for all of this in the discussion - but the insistence that you "know" "for a fact" that you are right with this, and the KRACH is just a load of garbage, is where it rubs me the wrong way.

BearLover
I posted betting odds because I wasn't aware of an actual NHL prediction model that gave probabilities for individual games. I've since found one, and it turns out I was wrong about the upper bounds of hockey probabilities

My site, hockey-reference.com, publishes Win Probabilities every day. As I mentioned upthread, it's based upon SRS, which is similar to KRACH, but takes into account goal differential.[/quote]

BearLover
Yes, this is true. But is the gap between Cornell and Union/Harvard as big as the largest gap between any two NHL teams?

Doubtful, but I don't think anyone here said it was. We also, however, don't know that, either way.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: adamw (---.phlapa.fios.verizon.net)
Date: March 03, 2018 07:12PM

jfeath17
One final point which I think has been established at this point, but I want to make sure we are all on the same page. Predictive models are going to have some subjectivity built into them. It is great that KRACH has no subjectivity and is a mathematically pure ranking for its goal of NCAA seeding since that needs to be "fair" and should be based on the actual outcomes of the games. However when creating a predictive model, we unfortunately do not have the luxury of a mathematically pure system. There are parameters and methods that must be chosen both when designing a predictor and measuring the performance. It is the designers goal to choose these such that the predictor is not over/underfit or have any bias's built in.

Absolutely. That is why I think it's a mistake to be so vehement about some better model, and so dismissive of what the current model is doing. There is something better, I'm sure, but exactly what is debatable.

BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: KGR11 (---.dyn.optonline.net)
Date: March 03, 2018 08:26PM

jfeath17
While you can't perfectly verify a prediction model, you can get an idea of its performance by separating the past data into training and testing sets. The fact that we are trying to predict probabilities and not simple classification does make it much more difficult to evaluate the performance. For classification problems the predictor is either right or wrong so it is easy to state a accuracy percentage. We however cannot directly observe the outcome probability of some matchup but only outcome of one trial of this matchup. This brings us to what I am attempting to do. By looking at the outcomes of many games with a similar krach predicted winning percentage we can come up with an estimate for the actual winning percentage of a team in this matchup.

My methodology for this was to use a gaussian weighted average of the games centered at varying krach probabilities. I calculated the winning percentage using these weights. I also used the weights to come up with average krach probability (this doesn't necessarily line up with the center of the gaussian particularly at the endpoints where all the games are to one side or the other). This is basically the logical extrapolation of the binning that was suggested earlier in the thread. The binning was actually the first analysis I did but the data didn't look good. I think the major improvement here is not that I am using the gaussian to come up with weights (that is probably overkill), but that I am using the weighted average of the krach probabilities rather than the center of the bin. What this trend line looks like can be changed significantly by changing the std dev of the gaussian (effectively changing the bin size). Basically the larger the bin the more underfit and the smaller the more overfit.


I also sought to measure the performance of KRACH in another way by looking at the R2 (Coefficient of Determination) The Wikipedia page is a pretty good explanation of this. The R^2 value can be looked at as the percentage of variance in the dependent variable (game outcomes) that can be explained by the independent variable (krach probability, etc..). These probabilities are all very low which makes sense since there is a lot of variability in the outcome of hockey games.

Independent Variable | R^2
--------------------------------------------------
Krach Probability | 0.023
Logistic Regression | 0.100
Linear Fit on Gaussian Average (0.1) | 0.112
(y=.749x+.126)


Another improvement I made was to include the inverse of each game (prob = 1-prob and swap wins and losses). This improves the fit around 0.5 since it is a little nonsensical if the matchup of two equal KRACH teams is not 0.5 in a model only dependent on KRACH.


One final point which I think has been established at this point, but I want to make sure we are all on the same page. Predictive models are going to have some subjectivity built into them. It is great that KRACH has no subjectivity and is a mathematically pure ranking for its goal of NCAA seeding since that needs to be "fair" and should be based on the actual outcomes of the games. However when creating a predictive model, we unfortunately do not have the luxury of a mathematically pure system. There are parameters and methods that must be chosen both when designing a predictor and measuring the performance. It is the designers goal to choose these such that the predictor is not over/underfit or have any bias's built in.

Many thanks to jfeath for this awesome work. One think I'd be curious to know is how the R-squared for the KRACH, Logistic Regression, and Linear Fit on Gaussian Average varies from year to year. This could give an indication of the extent (if any) that KRACH is a lesser predictor than your outputs.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jfeath17 (---.bstnma.fios.verizon.net)
Date: March 03, 2018 08:35PM

adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jfeath17 (---.bstnma.fios.verizon.net)
Date: March 03, 2018 08:37PM

KGR11
Many thanks to jfeath for this awesome work. One think I'd be curious to know is how the R-squared for the KRACH, Logistic Regression, and Linear Fit on Gaussian Average varies from year to year. This could give an indication of the extent (if any) that KRACH is a lesser predictor than your outputs.

I can try this out to just as a sanity check to make sure the models aren't overfitting. I may even do it on a completely new season to be sure.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: adamw (---.phlapa.fios.verizon.net)
Date: March 03, 2018 08:38PM

jfeath17
adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126

Wait, that's English? :) ... If you want to work with me on something going forward, feel free to drop me a line. adamw@collegehockeynews.com (same for anyone else who has chimed in here with something concrete to offer)
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jkahn (---.hsd1.il.comcast.net)
Date: March 03, 2018 10:28PM

adamw
jfeath17
adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126

Wait, that's English? :) ... If you want to work with me on something going forward, feel free to drop me a line. adamw@collegehockeynews.com (same for anyone else who has chimed in here with something concrete to offer)
Basically, what's being suggested here is to use KRACH for 75% of the probability and split the other 25% equally. I was actual thinking of suggesting a lesser dampening effect, such as using 90% KRACH and then adding that to 5% for each team, just based upon the feeling that even the weakest team should have at least a 5% chance - but that's just based upon gut feel, no data analysis.
Nevertheless, I do appreciate and enjoy the KRACH model, and do believe it's the fairest way of ranking teams.

 
___________________________
Jeff Kahn '70 '72
 
Re: 2018 ECAC Permutations
Posted by: Swampy (---.ri.ri.cox.net)
Date: March 03, 2018 10:44PM

adamw
What future probabilities of any kind are verifiable?

Well, if we have a fair coin and flip it repeatedly, we can calculate the probabilities before they happen.

This hinges on the coin being "fair." We can decide this by measuring its dimensions, the smoothness of its surfaces, and the uniformity of its density and metallurgic composition.

Of course, none of these measurements can be without some error. But if the errors are not biased high or low, this shouldn't matter.

We believe dimensions, smoothness of surface, and uniformity of density and metallurgy are important because we understand gravity, the mechanics of flat bodies, etc. We understand such things they've been studied in a wide variety of contexts, not only coin flipping, and these understandings have been corroborated by experimental results. Moreover, laboratory experiments are "closed" in the sense that researchers control not only the conditions for observations but also can intervene actively in experiments to create desired conditions for observation (e.g. throw coins with varying strength).

In contrast, there are two problems with hockey games. 1) We do not have a credible theory of hockey that explains why teams win or lose. 2) Games occur in "open" settings, where what goes on is not under experimental control and contingent on things that themselves may have any influence.
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: abmarks (---.hsd1.ma.comcast.net)
Date: March 03, 2018 10:49PM

jkahn
adamw
jfeath17
adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126

Wait, that's English? :) ... If you want to work with me on something going forward, feel free to drop me a line. adamw@collegehockeynews.com (same for anyone else who has chimed in here with something concrete to offer)
Basically, what's being suggested here is to use KRACH for 75% of the probability and split the other 25% equally. I was actual thinking of suggesting a lesser dampening effect, such as using 90% KRACH and then adding that to 5% for each team, just based upon the feeling that even the weakest team should have at least a 5% chance - but that's just based upon gut feel, no data analysis.
Nevertheless, I do appreciate and enjoy the KRACH model, and do believe it's the fairest way of ranking teams.

You are implying that there was a subjective choice of dampening effect?

Isn't the suggested formula there because it's the equation that defines the result of the logistic regression between KRACH and actual win% (when KRACH is grouped into buckets)?
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jkahn (---.hsd1.il.comcast.net)
Date: March 03, 2018 11:05PM

abmarks
jkahn
adamw
jfeath17
adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126

Wait, that's English? :) ... If you want to work with me on something going forward, feel free to drop me a line. adamw@collegehockeynews.com (same for anyone else who has chimed in here with something concrete to offer)
Basically, what's being suggested here is to use KRACH for 75% of the probability and split the other 25% equally. I was actual thinking of suggesting a lesser dampening effect, such as using 90% KRACH and then adding that to 5% for each team, just based upon the feeling that even the weakest team should have at least a 5% chance - but that's just based upon gut feel, no data analysis.
Nevertheless, I do appreciate and enjoy the KRACH model, and do believe it's the fairest way of ranking teams.

You are implying that there was a subjective choice of dampening effect?

Isn't the suggested formula there because it's the equation that defines the result of the logistic regression between KRACH and actual win% (when KRACH is grouped into buckets)?
No, all I'm saying is that my gut feel is subjective.

 
___________________________
Jeff Kahn '70 '72
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: jfeath17 (---.bstnma.fios.verizon.net)
Date: March 03, 2018 11:08PM

jkahn
adamw
jfeath17
adamw
BTW - I can't read your charts, so I have no idea what they're telling me. If there's an English translation, feel free.

My one sentence summary is that the current KRACH probabilities are biased towards the higher ranked team, a very simple modification which would greatly increase the accuracy is to change the formula to P(A Winning) = .749*(KRACH_A/(KRACH_A+KRACH_B)) + .126

Wait, that's English? :) ... If you want to work with me on something going forward, feel free to drop me a line. adamw@collegehockeynews.com (same for anyone else who has chimed in here with something concrete to offer)
Basically, what's being suggested here is to use KRACH for 75% of the probability and split the other 25% equally. I was actual thinking of suggesting a lesser dampening effect, such as using 90% KRACH and then adding that to 5% for each team, just based upon the feeling that even the weakest team should have at least a 5% chance - but that's just based upon gut feel, no data analysis.
Nevertheless, I do appreciate and enjoy the KRACH model, and do believe it's the fairest way of ranking teams.

Yes this exactly. Much better of a simplification than mine. :)
 
Re: 2018 ECAC Permutations - ECAC tournament draw
Posted by: BearLover (---.nycmny.ftas.verizon.net)
Date: March 04, 2018 01:23AM

adamw
Actually, I think you're the only one conflating anything, because I never said "worse" - so I'm not sure where this is coming from.

adamw
It is not certain that looking at things beyond wins and losses is any better. Goal differential has major flaws, and might not mean much. Shot differential has its own issues, but could be a decent factor. Honestly, I'm not all that interested in things like goal and shot differential.

So you wrote an entire post taking issue with how I put words in your mouth that you said shot differential/goal differential is a worse predictor than wins/losses, when in actuality you were merely saying both are equally flawed? Then just replace "worse" in my post with "just as bad" and my points still stand.

adamw
What has rubbed me the wrong way with your posts is your unnecessary (to me) vehemence against KRACH, the twisting of what I've said, and the high-level of self-confidence in what you're saying. A little humility is helpful here because no one really knows.
Oh I'm fully aware I don't really know anything. I don't have a background in statistics or mathematical modeling (or swimsuit modeling). I just watch a lot of hockey and read a lot about hockey, and nothing I've seen or read suggests a predictive model based entirely on win % over a 30-game season is going to be very accurate. You/CHN have done a convincing job arguing that KRACH is built to be (almost?) the best model for measuring past success. It isn't built to predict the future, though.
Edited 1 time(s). Last edit at 03/04/2018 01:49AM by BearLover.
 
Page: Previous1 2 
Current Page: 2 of 2

Sorry, only registered users may post in this forum.

Click here to login