Probabilistic, Iterative Player Rating Tool

[skyking162]

Long time, no post. Hi, RJ!

Back when Nate Silver was at Baseball Prospectus, he mentioned an Excel player rating tool he had for personal use that used a probabilistic, iterative approach. I want it.

Somehow, you'd create probabilities of each player being selected at each pick in the draft. (Maybe start with ADP data, smooth it out?) Then update these probabilities after each pick. This would allow you not just to use a marginal value system based on last player picked, but also on gaps between tiers at each position and number of picks before you're up again.

Has anyone seen something like this? Anyone have further ideas on how to create one?

[DJBeasties]

Welcome back dude!

[joncarlos]

I've never seen this, but I'm definitely intrigued by it. I was looking at my fantasy hockey spreadsheet, which is only 3 positions (F, D, G) and has VBD ratings calculated for each player. It would be a much simpler exercise than doing a baseball draft with 10+ positions. So basically something like this:

PLAYER POS VBD
Sidney Crosby F 5.29
Jonathan Quick G 5.25
Evgeni Malkin F 4.84
Henrik Lundqvist G 4.61
Dustin Byfuglien D 4.36
Steven Stamkos F 4.27

How do you figure out the circumstances under which a drafter wouldn't just take the next highest rated player by VBD? You'd need to take replacement level into account, as well as what the drafter already has.

I'll give it some more thought. I am intrigued but other than doing it because it's interesting, I'm not sure you could get it right enough to justify all the work you had to put into it. Unless you were as smart as Nate Silver, of course.

[onejayhawk]

I've never seen this, but I'm definitely intrigued by it. I was looking at my fantasy hockey spreadsheet, which is only 3 positions (F, D, G) and has VBD ratings calculated for each player. It would be a much simpler exercise than doing a baseball draft with 10+ positions. So basically something like this:

PLAYER POS VBD
Sidney Crosby F 5.29
Jonathan Quick G 5.25
Evgeni Malkin F 4.84
Henrik Lundqvist G 4.61
Dustin Byfuglien D 4.36
Steven Stamkos F 4.27

How do you figure out the circumstances under which a drafter wouldn't just take the next highest rated player by VBD? You'd need to take replacement level into account, as well as what the drafter already has.

I'll give it some more thought. I am intrigued but other than doing it because it's interesting, I'm not sure you could get it right enough to justify all the work you had to put into it. Unless you were as smart as Nate Silver, of course.

It is not smart so much as fixated. As a math major, I knew guys like Silver. Once they got an idea in their head, they would worry at it like a dog on a bone.

J

[skyking162]

How do you figure out the circumstances under which a drafter wouldn't just take the next highest rated player by VBD? You'd need to take replacement level into account, as well as what the drafter already has.

Here's an example. You're up, and then you pick again in 10 picks. Shortstops on the board have ratings of 26, 22, 20, 15, 10, 5, 0. Second basemen have 28, 27, 24, 22, 12, 7, 2. The best 2B is the highest rated, but there are two other 2B's better than the second-best SS. So taking the 26 SS now is a good move if there's a good chance the 27 or 24 2B will be around in 10 picks.

[joncarlos]

Here's an example. You're up, and then you pick again in 10 picks. Shortstops on the board have ratings of 26, 22, 20, 15, 10, 5, 0. Second basemen have 28, 27, 24, 22, 12, 7, 2. The best 2B is the highest rated, but there are two other 2B's better than the second-best SS. So taking the 26 SS now is a good move if there's a good chance the 27 or 24 2B will be around in 10 picks.

So you basically want something called VORP10 (for 10-team drafts), where the players have a set VORP for the whole draft, but also a moment-specific VORP.

SS that will go in the next round are 26, 22, 20, so if you wait, you'll get a SS of 15. The first guy then has a VORP10 (in the next 10 picks) of 9 (26-15).

2B that will go in the next round are 28, 27, 24, so if you wait, you'll get a 2B of 22. The first guy has a VORP10 of 6 (28-22).

Basically looking for cliffs and plateaus.

[jackmadden]

You can calculate a player's probability of being at a given pick in excel using the formula 1 - NORMDIST(p, a, s, true). Where 'p' is the given draft pick, 'a' is the player's ADP and 's' is the standard deviation for the player being drafted.

[skyking162]

You can calculate a player's probability of being at a given pick in excel using the formula 1 - NORMDIST(p, a, s, true). Where 'p' is the given draft pick, 'a' is the player's ADP and 's' is the standard deviation for the player being drafted.

Is that true? Is pick expectation symmetric? Consider Albert Pujols. He has a significant likelihood of going 1/2/3 (right?) and then it tails off. He can't go 0th or -1st.

Guys in the 4th round probably start as nearly symmetric, but as you get into the 3rd round and they aren't picked, you'd want to update their distributions and they'd start to look like Pujols' -- sooner rather than later. (I think.)

[bryanbutler]

i update my player values as every player is auctioned off, by adjusting the VORP on-the-fly, and then sort by positions in a printout on the screen to see with my eye where the plateaus are. but that's not quite what you're asking for. we have one guy in our league that has software that does almost exactly what you're talking about, though it's not in excel. it does a bayesian analysis of the improvement that a given player would give him compared to what other players are left, positions open, etc., then given how much money he (and everybody else) has it tells him how much he should spend. that's what you get when you have a league full of caltech PhDs . but this is for auctions, not drafts. (rotolab and similar software probably does something similar...)

the problem with just using a formula as proposed by jackmadden is that it doesn't account for position scarcity. as JC said, you really need to add in the position filling. in fact, i don't think you care about ADP at all. i think you need to know what the value of every player is to your roster (which changes as you, and other GMs, take players). and you need some concept of position scarcity as each player is taken and rosters/positions are filling up. then, knowing how many picks it is until your next pick, you can make the best choice of who to take.

FWIW, i wouldn't worry about the non-symmetry of the distribution for the very early picks if you did it the simple way. i'd guess that the standard deviations of the early players are small enough that once you're beyond the first handful of players you won't ever violate the symmetry assumption. you could look at positive-definite probability distributions if you really wanted to, but i personally don't think it's worth the effort.