How to compare a forecast with probabilities and a forecast with quantities

At Advance Auto Parts, we have replaced an old legacy forecast, which was creating a probability to sell, with a new forecast, which predicts a quantity to be sold. While doing this, we had to compare the two forecasts, in order to demonstrate that the new forecast is indeed better than the old one. This talks describes how we did the comparison and what kind of pitfalls we ran into.

Julia Kuznetsova works as a Data Scientist at Advance Auto Parts. Julia has a Ph.D. in Linguistics with a focus in computational linguistics; she wrote and published a research monograph “Linguistic Profiles: from Form to Meaning via Statistics” (2015), which investigates correlation between distributional and semantic properties of linguistic units. Julia is a Program Director for Data Scientista Society – a meetup for women in Data Science.

quantities