Ladies and Gentlemen, welcome to the coal face of insurance analytics.
Today’s guest is Jim Weiss, the director of analytic solutions for ISO. ISO houses perhaps the richest insurance data repository on the planet and among Jim’s responsibilities is building models that don’t use it! I joke.. Actually, Jim is exploring new frontiers of modeling for insurance purposes. This episode works very well in conjunction with the Cathy O’Neil episode which of course I recommend you listen to right away!
Right out of the gates Jim discusses his view on whether big data in insurance is overrated or underrated:
Jim Weiss: I think if you were gonna write a history of big data in insurance ending today, I would probably have to say that maybe big data in insurance is a little bit overrated. If you look at our industry in recent years, I think it’s kind of a graveyward of big data and analytics type projects that went overbudget overdeadline. I think there were a lot of factors contributing to that, but they can largely be characterized as maybe we’re not doing these projects very well but moreover we’re not picking these projects very well.
Jim on proxies:
Jim Weiss: I feel it’s very difficult to identify something to predict risk behavior that isn’t a proxy unless you’re doing individual risk rating. Unless you’re using something like prior claims to predict future claims, what variable isn’t proxy-ing for something? [From there we talk about my own history as a teenage driver (not great)!]
Jim on how good we are at what we do:
Jim Weiss: Myself and a colleague did a study of some rate level reviews that had been conducted in our industry over the past several years to see how many of them reversed themselves over one or two years.
David Wright: change signs.
Jim Weiss: Not even change signs, substantially reverse themselves. So you have a plus five rate percent rate indication. You notify your agents, you put it in your systems, you sent out hte policy holder notices. you tell the regulators you do eerything you ahve to do. you spenda ll that money, time and effort. then one or two years later. boom, negative five. Completely reversed…
Luckily in the study we did, the majority of the time at the time the rate level indications didn’t reverse themselves within one or two years. It did happen a little bit more often than perhaps I would think.
DW: why do you think that would have happened.
JW: because, Yogi Berra has an expression that making predictions is hard, especially about he future. Ther are so manythings you don’t know at the time you make the analysis.
Finally, in my favorite part of a conversation full of big insightful moments, we discuss whether and how to use complex modeling (and what on earth is modeling for?!):
Jim Weiss: I think, to some degree, applying the smell test to the types of variables you’re looking at, ‘is there some basis in reality’, can be a healthy thing, but I don’t htink it should be preclusive of exploring more complex approaches where prudent. But I’m not sure pricing is necessarily the prudent place for it. It may be but there are a lot of use cases for big data and analytics and sophisticated techniques in our industry which far transcend pricing.
Jim Weiss: the mix of complicated problem and complicated solution is a particularly problematic one… if you don’t really understand why exactly the approach you’re taking does solve that problem then how do you know it’s not a coincidence.
David Wright: I’m wondering what the objective is of modeling. So one characterization of the objective of modeling is to get an answer you can use. So I get a rate, or I get a loss estimate. I wonder if the real objective is to develop an understanding of the problem. Which is a human consumable… So the output isn’t the answer the output is the story.