An aside on the notion of streakiness (part 1B of the series, if you will).
There have been many studies showing that all streakiness in sports is random. There was an exhaustive study a couple of years ago on Retrosheet that found that hot and cold streaks had no predictive power.
The problem is in the assumption that if a hot streak or cold streak is real, it must have significant predictive power.
In baseball, the standard test of the reality of hitting streaks is serial correlation. Is a player’s performance in one game predictive of his performance in the next? The problem with this is that there’s an enormous amount of “noise” added to the signal. A hot hitter will face Sabathia and go 0-4, a cold one will face a AAA callup and / or get two bloop hits. Red Sox and Yankee fans may remember a series in NYC (May 27-29, 2005) where Manny Ramirez, one of baseball’s truly streaky hitters, came in 1 for his last 12 and looking awful and went 7-13, each and every one a cheap single; he then left town and put up a 562 OPS in his next 10 games.
The further problem is that in a serial correlation, the end of each streak and the beginning of the next form a pair of points included in the correlation, when our hypothesis is that they should anti-correlate. That further reduces the strength of the measured correlation.
In fact, if you take Manny’s career with Boston and divide it into apparent hot (actually just normal) and cold streaks and remove the anti-correlated data pairs, you do get a significant or nearly significant serial correlation (of linear weights / PA). And as I have noted elsewhere, player seasons often divide into chunks that chi-square tells us are unlikely to be random.
Standard statistical tests of streakiness just aren’t up to the task of demonstrating it’s real. That doesn’t mean it isn’t real — a perfect example of what Bill James calls the “fog.”
I’m 100% certain that a study using experienced baseball scouts could prove the existence of streakiness by having them significantly outperform chance in their ability to predict the end of slumps by streaky players like Manny (as Jerry Remy used to do). IOW, they’d say, “OK, today player X fixed his mechanics and should perform better over the next N games than the last N.” And they would be right most of the time.