Every Rookie Makes The Same Mistake

“They were running the biggest start-up in the world, and they didn’t have anyone who had run a start-up, or even run a business,” said David Cutler, a Harvard professor and health adviser to Obama’s 2008 campaign, who was not the individual who provided the memo to The Washington Post but confirmed he was the author. “It’s very hard to think of a situation where the people best at getting legislation passed are best at implementing it. They are a different set of skills.”

That about sums up what the whole Health Exchange fiasco is about. Who put who in charge of this, anyway?

In the end, the economic team never had a chance: The president had already made up his mind, according to a White House official who spoke on the condition of anonymity in order to be candid. Obama wanted his health policy team — led by Nancy-Ann De­Parle, director of the White House Office of Health Reform — to be in charge of the law’s arduous implementation. Since the day the bill became law, the official said, the president believed that “if you were to design a person in the lab to implement health care, it would be Nancy-Ann.”

More here. I have this theory that everyone makes the same decision the first time they come upon an IT project. Can’t be that hard, right? Find someone you trust and put them in charge.

And there’s this from an MR comment:

A lot of focus has been on the front-end code, because that’s the code that we can inspect, and it’s the code that lots of amateur web programmers are familiar with, so everyone’s got an opinion. And sure, it’s horribly written in many places. But in systems like this the problems that keep you up at night are almost always in the back-end integration.

The root problem was horrific management. The end result is a system built incorrectly and shipped without doing the kind of testing that sound engineering practices call for. These aren’t ‘mistakes’, they are the result of gross negligence, ignorance, and the violation of engineering best practices at just about every step of the way..

Is The Stock Market Overvalued or Not?!

I’m too much of an efficient markets guy for that title to be entirely serious but today I’ve seen two interesting graphs and read one really deep blog post that are making me think. Here is graph 1:

If you calculate a price/earnings ratio using annual data, then in a dismal economic year like 2008 when profits are very low, the P/E ratio will spike dramatically. To avoid these somewhat meaningless short-term spikes, the Shiller P/E ratio looks the current price of stocks divided by the average profit levels over the previous 10 years, so that it is less influenced by economic conditions this year.

The Shiller P/E is now 24.8. As the figure shows, it is higher than at any time except the peak of the dot-com boom and its aftermath, and Black Tuesday back in 1929 at the front edge of the Great Depression. In other words, when the P/E ratio has reached this level in the past, sometimes it has gone still higher (as in the dot-com boom), but over the last 130 years it has then always fallen back.

Shiller PE Ratio Chart

The point here is that, using the price to the 10-year average profit, the stock market looks massively overvalued from a historical perspective. That post also points us to what I’ll call the Hussman post, which has a fascinating discussion on the bear case and follows up with an even better graph that takes the first one and shows (in the red line) that previous overvaluations meant horrible stock returns and vice versa.

I don’t get it. We’ve just had a lost decade where the valuation never ‘corrected’ to the levels seen between ’72 and ’90 and now we’re climbing again? Hussman also has this to say:

On careful analysis, however, the clearest and most immediate event that ended the banking crisis was not monetary policy, but the abandonment of mark-to-market accounting by the Financial Accounting Standards Board on March 16, 2009, in response to Congressional pressure by the House Committee on Financial Services on March 12, 2009. The change to the accounting rule FAS 157 removed the risk of widespread bank insolvency by eliminating the need for banks to make their losses transparent. No mark-to-market losses, no need for added capital, no need for regulatory intervention, receivership, or even bailouts.

I didn’t even know that. Anyway, here’s the next graph:

wealth

Suddenly everyone got wealthier? Price increases do ‘happen’, I guess. But why?

Back to the Hussman post for this quote:

The predictable contraction in corporate profit margins will certainly contribute, but remember that changes in corporate profits typically follow changes in combined government and household savings with a lag of 4-6 quarters, and most of the recent shift in combined savings has only occurred since the third quarter of 2012.

And now we’re at the point where I realize why I’m so lost. I don’t understand P/E ratios but I do understand profits and can see why lower consumption levels probably do mean lower profits in the short term, particularly since those higher savings levels are probably chipping away at big debts. Let’s just take that one at face value or now.

What irritates me is that those charts above tell me that I should want to live in an era with low P/E ratios. Do I?

One more chart, graphing annual corporate earnings (source):

S&P Earnings

One thing you might notice about the 80s is that that corporate earnings curve is basically totally flat. Does that sound as appealing to you? Your employer not growing?

And what is the point of the 10-year p/e, anyway? Why 10 years? Looking back at that graph of the 80s at the top, showing the 10-year return being the inverse of the valuation level, selling 10 years after the bottom of a market normally puts you square into the top of another market. And the longer those ‘tops’ last (in this case 10 years) the more ‘people’ who bought into those prolonged ‘bottoms’ sell out.

There could be a lot of bias in that 10-year measurement.

So here’s where I am:

  • I don’t understand why market valuations grew so much at the end of the 90s. I’m not afraid to climb into my armchair and say I like to think that the baby boomers had something to do with it. Could also be the great stagnation biting down since nonproductive investment defines ‘bubbles’.
  • I don’t understand why corporate profits were stagnant in the 80s, grew in the 90s and have been stagnant in the 2000s. Great stagnation?
  • I don’t understand why nobody bothers to talk about profits when analyzing the stock market as a whole. Isn’t that the point of all this?
  • I don’t understand what monetary policy has to do with any of this.
  • I definitely don’t understand what any of this means for investing. Hussman says stocks and long term treasuries are about roughly the same bet at the moment. A common prediction.

So, once again, I have seen some really neat stuff on investing and come away with nothing. How do people do this for a living?

Your Computer To Teach You

Here is a 2011 Kurt VanLehn paper (pdf) on human vs. computer systems of tutoring:

This article is a review of experiments comparing the effectiveness of human tutoring, computer tutoring, and no tutoring.  “No tutoring” refers to instruction that teaches the same content without tutoring.  The computer tutoring systems were divided by their granularity of the user interface interaction into answer-based, step-based, and substep-based tutoring systems.  Most intelligent tutoring systems have step-based or substep-based granularities or interaction, whereas most other tutoring systems (often called CAI, CBT, or CAL systems) have answer-based user interfaces.  It is widely believed as the granularity of tutoring decreases, the effectiveness increases.  In particular, when compared to No tutoring, the effect sizes of answer-based tutoring systems, intelligent tutoring systems, and adult human tutors are believed to be d = 0.3, 1.0, and 2.0 respectively.  This review did not confirm these beliefs.  Instead, it found that the effect size of human tutoring was much lower: d = 0.79. Moreover, the effect size of intelligent tutoring systems was 0.76, so they are nearly as effective as human tutoring.

One more specific result found in this paper is simply that human tutors very often fail to take advantage of what are supposed to be the advantages of human tutoring, such as flexibility in deciding how to respond to student problems.

That’s from MR. One way of thinking about this is that human instruction has a much higher variance but that in the future teachers whose skills fall below some threshold will find students switching to machines. One key will be a fair test for comparison and I worry about anti-machine prejudice.

The bottom line, of course, is that the most important ingredient in learning is motivation and the best humans will always be better motivators than machines. Only they can invoke presence with charisma and clear, relatable explanations.

Don’t count the motivating power of machine-centered instruction out totally, though. Gamification techniques feel mostly immature (at scale) and could potentially tap the competitive instincts of some.

Because Science Got Bigger

“Both inside the humanities and outside, people feel that the intellectual firepower in the universities is in the sciences, that the important issues that people of all sorts care about, like inequality and climate change, are being addressed not in the English departments,” said Andrew Delbanco, a Columbia University professor who writes about higher education.

That’s the NYT on the decline of humanities majors.

Many years ago my industry, insurance, probably employed lots of non-technical people in analytical and sales roles. The skills required were more generalist because the business was more personal.

But that’s all changing. Insurance has gotten miles more scientific in the last 20 years and that has driven all the trends that we spend our time talking (complaining) about: the rise of actuaries, industry consolidation, margin squeezing, etc.

Veterans spend a fair bit of time missing the old days and in this case it’s true: they were very different. But as our tools for measuring all sorts of things have improved, the social purpose of our industry (risk management) is being being better achieved.

Assess a Coffee Shop in One to Fifteen Seconds

I’m a coffee snob. Bad coffee tastes like garbage and I don’t like the idea of chaining cups together to dodge the caffeine hangover. So I’m perfectly happy with nothing unless I can get an outstanding cup of joe.

In fact, I’m somewhat embarrassed to say, I’d probably order a bottle of water in 99% of the coffee shops on earth. And up until recently that included 100% of the coffee shops in the financial district of NYC where I work.

I’m pleased to report, though, that the coffee revolution is breaking over this city before my eyes and I thought I’d celebrate with a little guide to figuring out if you’ve found yourself in a spot with incredible coffee.

Step 1: look through the door/window. This step will cut out about 98% of the coffee shops you’re likely to run across. You’re looking for hardware, specifically these two items:

coffee2The red circle is showing you a bean grinder (sorry about the quality of the image – this is from my ipod camera at Bluestone which just opened near my office). Specifically, it’s a burr grinder. All high end coffee shops have them and they signal two very important things:

  1. The beans are ground fresh. This means they understand the basics of coffee.
  2. They are a bit obsessed about quality. You see, they aren’t using a blade grinder, which adds a little bit of heat to the beans as it grinds them. Personally I don’t think that this moves the needle too much in terms of flavor but it means that they care enough to spend the extra $300 or whatever it costs for a burr grinder.

Obsession with quality is a very, very important signal because in your 1-15 seconds you can’t observe one of the critical elements to awesome coffee: the quality and freshness of the beans. You need signals correlated with bean quality. And the burr grinder is one.

The green arrow points to the other important piece of kit: an incredibly large and expensive espresso machine. Now, I don’t care about whether you like espresso drinks or not (I happen to drink them almost exclusively). This machine signals two more things:

  1. They spent $10,000 on a coffee machine. More quality obsession.
  2. These things are hard to use properly. This means you might have a real barista on your hands. BUT you don’t know that yet.

So you need to evaluate the barista. If he/she sucks the it means the owner spent a ton of money but does not care about employing and training people passionate about coffee. In other words, you get an overpriced Dunkin’ Donuts.

But you can’t really get a good view from outside so to evaluate the barista this you’ll probably actually have to walk into the coffee shop. But you only have 5-10 seconds left, so what are you looking for? The tamp.

That little thing in the imaged woman’s left hand in is the basket. That’s what you grind the beans into. The tool in her right hand is the tamp, which is also the verb describing what she is doing. Note two things about this woman’s tamp:

  1. The basket is propped against the table.
  2. Look at the position of her arm: she is putting her shoulder into it!

The key to a great tamp is the pressure. About 30lbs, I’ve heard, will do it. Most poorly trained baristas half-ass the tamp. Sadly enough, a half-assed tamp can wreck even the best espesso pulled from the best machine.

Now there are many many other steps that go into great coffee than what I’ve listed here. And none of these steps I’ve mentioned above, strictly speaking, are necessary for excellent drip coffee. But the point is that you can suss out every one of the above-mentioned signals in a matter of seconds in any coffee shop you choose.

And if your shops passes these tests, you can be pretty sure anything you get from them is going to be fan-freaking-tastic.

Problems that Bug Me: The n-1 Correction for Sample Variance

Let me start with an easy example that everyone starts with in stats: we want to know how variable height is in the population of humans. We can’t measure the height of all humans so we gather up the few around us and here’s what we get.

Janet: 5ft
Sam: 6ft
Shaq: 7ft.

The average height of the three is: \frac{5+6+7}{3}=6.

The variance of height is: \frac{(5-6)^2+(6-6)^2+(7-6)^2}{(3-1)}=1

So the statistician says we now have an estimate for average height and variance of heights of the population of humans: 6ft tall with a variance of 1.

Here’s a twist, though. If there were only three humans in existence, the variance would be different: \frac{(5-6)^2+(6-6)^2+(7-6)^2}{3}=\frac{2}{3}.

So what’s the difference? The difference is that if Janet, Sam and Shaq are a sample of the population we divide the sum of squared differences from the mean by n-1=2. If they are the entire population, we divide it by n=3. This makes the sample variance larger than the population variance. Why is that?

This is a question that annoyed me for some time. The key, though, is to not focus on the variance calculation. The key is the mean.

You see, when you pull out a sample of a population you aren’t measuring the correct average height. Your height estimate is going to be wrong and it’s going to be different each time you draw a sample from the population. In other words, it’s going to vary and so increase the variance.

So the point of the n-1 correction is to increase the variance estimate to allow for the fact that your measurement of the mean varies with each possible sample.

If the sample gets big enough, dividing by n-1 isn’t much different than dividing by n. Imagine the difference between dividing by 100,000 or 100,001. So this correction becomes meaningless because the mean we are measuring is probably the correct one.

Here’s the math:

Let’s start with the sum of squared errors, which is this: \sum\limits_{j=1}^k(Y_j-\bar{Y})^2. Remember that \bar{Y} is the mean of the sample group. What we’re going to find is that it’s equal to this: (k-1)\sigma^2

which is the same thing as saying this E[\frac{1}{(k-1)}\sum\limits_{j=1}^k(Y_j-\bar{Y})^2] = \sigma^2.

\sum\limits_{j=1}^k(Y_j-\bar{Y})^2 = \sum\limits_{j=1}^k[(Y_j-\mu) + (\mu-\bar{Y})]^2. We start with one of the oldest tricks in the book. Adding and subtracting the same amount from the equation.

= \sum\limits_{j=1}^k[(Y_j-\mu)^2 + 2(Y_j-\mu)(\mu-\bar{Y})+ (\mu-\bar{Y})^2]. Expand the square.

= \sum\limits_{j=1}^k(Y_j-\mu)^2 + 2(\mu-\bar{Y})\sum\limits_{j=1}^k(Y_j-\mu)+\sum\limits_{j=1}^k(\bar{Y}-\mu)^2. Split up the summations.

= \sum\limits_{j=1}^k(Y_j-\mu)^2 + 2(\mu-\bar{Y})(k\bar{Y}-k\mu)+k(\bar{Y}-\mu)^2 Recognize that a sum of means is k*the mean.

\sum\limits_{j=1}^k(Y_j-\bar{Y})^2 = \sum\limits_{j=1}^k(Y_j-\mu)^2 -k(\bar{Y}-\mu)^2. Simplify a bit. This is a pretty key step, actually, because now we see that the sum of squared error (the left hand side, which I’ve restated here for clarity) is smaller than the sample squared errors using the true mean, mu.

E[\sum\limits_{j=1}^k(Y_j-\bar{Y})^2] = E[\sum\limits_{j=1}^k(Y_j-\mu)^2 -k(\bar{Y}-\mu)^2]. Take the expectations. Boy, don’t those look like variances?

= \sum\limits_{j=1}^kVar(Y_j) -kVar(\bar{Y}) Yep.

= k\sigma^2 -k(\frac{\sigma^2}{k}) = (k-1)\sigma^2 The home stretch.

E[\frac{1}{(k-1)}\sum\limits_{j=1}^k(Y_j-\bar{Y})^2] = \sigma^2 and Done.

Addicted to… something

Still entrenched in the CrossFit culture of deplete, endure, repeat, she quieted the alarms and stoically pressed on to go to work. It didn’t take long to realize she not only couldn’t bend her arms, they also had no strength. She wasn’t able to treat her patients. By that evening, her slender arms had continued to swell into plump hotdogs of ache and regret, and she was starting to come to the realization that the morning’s danger alarms were legitimate.

Unbelievably, it took another 24 hours for her professional sense to break through the grip of the CrossFit culture, and seek medical attention. She was diagnosed with acute rhabdomyolysis, and ended up in the hospital for over a week. While in the emergency department they tested her creatinine kinase (CPK) levels. Normal is about 100. Her CPK levels were more than 45,000, a number that indicated damage to the kidneys.

While in the hospital, she called to cancel her CrossFit membership. As is standard when something is cancelled, the CrossFit coach asked the reason for her decision. She replied, “I’m in the hospital.” The instructor quickly asked, “Is it rhabdo?”

Rhadbo is short for Rhabdomyolysis and its description reads like something from a comic book (exploding muscle cells poisoning your kidneys, eventually killing you). It’s caused by consistent, extreme exertion.

The NYT ran an article on this in 2005:

Yet six months later Mr. Anderson, a former Army Ranger, was back in the gym, performing the very exercises that nearly killed him. “I see pushing my body to the point where the muscles destroy themselves as a huge benefit of CrossFit,” he said.

I think it’s safe to say that anyone that works out until their muscles explode isn’t playing with a full deck. These people are addicted to exercise. But isn’t working out a good thing? Well, for most people so is losing weight, yet we have anorexia.

One amusing consequence of these stories is that people might use it as an excuse for not working out at all. Or at least to make themselves feel better about not working out.

How Insurance Companies Die

If you’re curious, follow the news about Tower Group. Here’s David Merkel’s take:

As an analyst of insurance stocks, I was always skeptical of Tower Group for three reasons:

  1. The acquisitive nature of Tower Group.
  2. The rapid growth in premiums, 52% per year over the last 10 years — no insurance company can successfully grow that rapidly in a mature market.
  3. Odd reinsurance agreements that made me wonder.

The easiest way to grow an insurance company is to pretend your insurance policies cost less than they do, which is easier to pull off at the pointy end of the insurance market where all the strange coverages and specialty products are. In these lines, management uses a lot more judgment when picking reserve development and pricing factors. There isn’t enough data for actuaries to get a real firm grip on things.

As I understand it, this is where the pain in Tower Group is being felt. Management undoubtedly knew it was pushing things too hard, trying to grow its way out of its past problems. And you can do this for a long, long time at an insurance company. Years. Merkel’s comments above were made almost 10 years ago. And he was right. Yet that insight yielded no profitable investment opportunities, as he rightly saw. 

But like all downward spirals, it’s harder to get out the longer you’ve been doing it.

Toughest Accountant On Earth

Juan Manuel Marquez, a one-day Hall of Fame Boxer:

“I would run in the mornings, then work my 9 to 5 accounting job, and then to the legendary Romanza Gym to do my boxing work to prepare for my fights. When I had fights and had to travel I would get permission from by boss to go for my fights traveling on the weekends, fight, and be ready for work on Monday. I did that until I got my first championship opportunity against Freddie Norwood. At that time I took a few months off from work to get ready for the fight. A fight I felt I won on points. But it was not until I won my first world championship in February of 2003 that I decided to leave my job and concentrate on my boxing career. I was a little nervous about leaving my job and its security but I felt that as a World Champion I needed to dedicate myself to boxing fulltime and it was the right decision to make.

Not easy funding yourself as a professional athlete. Very few boxers make it so while I’m not surprised that he had a fall back plan but I am surprised at how long he kept at his day job!

You may have heard of Marquez for this (original post here).

The thing that impressed me about that fight was that Manny was outboxing Marquez but found himself in the ring with what might be the second smartest fighter on the planet. And smart with the figurin’, too!

Actuaries’ Achilles Heel

I spend a fair bit of time whining about how actuaries basically ignore the overfitting problem when building their models. So I was pretty pumped to see an actuary address this issue head on for stochastic reserving (pdf pg 29). I haven’t studied stochastic reserving models, but I was disappointed with this:

Let’s start with a collection of normal distributions with the mean, µ, being uniformly distributed between 100 and 200. The standard deviation, σ,is uniformly distributed between 25 and 50. Pick a random parameter set (µ,σ) from this collection. Then pick a random training sample, x1, x2, x3, x4 and a random holdout sample, x5, from a normal distribution with mean µ and standard deviation σ.

Dude… assuming normal?

I expect a common response to these simulations would be that the Bayesian models assumed I “knew” the correct prior distribution. My, typically Bayesian, response to that would be that if your prior distribution truly reflects your prior beliefs, you should believe the posterior result.

An immensely unsatisfying response. I get that it is frustrating for people to criticize fundamental assumptions behind somewhat sophisticated techniques. If you believe that overparametarization is real, the story goes, you must also reject the normal assumption and therefore you must reject a TON more theory on stochastic reserving, not just this little part. Fair enough, I say.

But what an incredibly weak defense.