Time Flies When You’ve Practiced Your Butt Off

Posted on by David Wright

One of these ideas that I like being hammered with over and over is that performance (skill) is about repetition. It’s about muscle memory for athletes and autopilot recall for exam-takers.

And this is what it’s about for Navy Seals:

The best preparation for any scenario is having done it before — or the closest thing to that.

Via Daniel Coyle’s excellent book The Little Book of Talent: 52 Tips for Improving Your Skills:

When U.S. Navy SEAL Team 6 mounted its May 2011 raid on Osama bin Laden’s compound in Pakistan, it prepared by constructing full-scale replicas of the compound in North Carolina and Nevada, and rehearsing for three weeks. Dozens of times the SEALs simulated the operation. Dozens of times, they created various conditions they might encounter. They used the power of repetition to build the circuitry needed for the job.

This is one of the key principles of deliberate practice, the best system for building expertise.

Another way of putting this is that these SEALs have planned and practiced reactions to every conceivable scenario. That way they can maximize how many situations they react to with ‘instinct’, which is just a way of saying they’re on mental autopilot. When trained properly, autopilot is simply a faster and more reliable mode of human thought than focused attention.

The cornerstone of my exam strategy is mental autopilot. I maximize what I can achieve without really thinking, saving my mental resources for the very end. Here is the progression I go through at exam time:

  1. Comb through the test and read every problem. If I know the answer INSTANTLY, I work the problem (category #1 problems). If I have any hesitation at all, I skip it. If I don’t complete more than half of the test on this pass, I am in big trouble. Time: 25-33% of total allotment (45 mins-1hr in a 3-hour test).
  2. Go to the problems (category #2 problems) that I think I can figure out quickly and skip the ones that are really tough. Most of these problems are easy problems disguised as really hard problems. Once I crack it, I switch on autopilot again and blow through the sucker. This should get me to about 75% complete. This is probably about another 25% of the exam time.
  3. Go back over every question and (re-)work them all. I’ve probably screwed up a few category #1 problems (which are actually category #2) and I correct these here. It takes discipline to keep skipping the really tough problems but I need to save those. Get the easy ones right!
  4. The real toughies should be <10% of the total problem set. Now you tackle them.  You’ve probably got about 20-30 mins left on the exam and your brain is nearly fried. These last few minutes will take FOREVER because once you switch off autopilot the world slows down.

This is why return journeys feel quicker, why childhood feels like it takes FOREVER and why time flies when you’re having fun. Thinking is HARD. Thinking is PAINFUL. Your brain only has a finite amount of focus and attention and lighting up all those neurons is costly.

When you are studying, your goal is to have an autopilot that is as complete as possible. Understand that to sign up for an exam is to sign up for mental torture: you need to think a LOT to learn and, as I said, thinking is PAINFUL. That’s ok: no pain, no gain and you’re here to make something of yourself, dammit.

If you don’t train properly, you need to do more thinking on the test, which is costly. You need to work dozens/hundreds/thousands of problems to have well worn grooves in your mind for the problems you encounter.

Generally speaking, kids have no grooves. Older folks have nothing but grooves. Neither group likes putting down those grooves; kids just have no choice in the matter.

Learning Other Languages

Posted on by David Wright

There have been a few arguments presented out there against teaching American students  languages other than English in school. Here’s Bryan Caplan:

If you find the typical American insufferably insular and low-brow, I agree.  My point is that given his insular, low-brow ways, the typical American who remains monolingual isn’t missing much.

Tyler Cowen responds by mentioning that the main benefit from learning another language is a general cognitive one. Cowen also refers to this New Yorker article, form which I learn of Larry Summers’ latest controversy. Here’s the key piece of some remarks he made recently:

English’s emergence as the global language, along with the rapid progress in machine translation and the fragmentation of languages spoken around the world, make it less clear that the substantial investment necessary to speak a foreign tongue is universally worthwhile.

When my wife and I were living in Toronto she had a job hiring and coordinating promotional reps across Canada for various brands. She, like all Canadians, took a lot of French in elementary and secondary school and even grew up on the Ontario/Quebec border. So what did she do when she needed to read emails from her French reps? She used Google Translate.

I grew up in a much more Anglo part of Ontario but studied in a French Immersion program in elementary school where just about everything was in French until I was 13 or so. At my peak I’d say I was totally fluent (more so than my wife) but I had lost most of it by my mid-20s. I could still read reasonably well, though, and felt that the Google translation was excellent. Very impressive.

If you evaluated all the time my wife and I spent learning French against the direct economic benefit it afforded us (and this is in Canada, remember), it’d be easy to dismiss multilingualism as a waste. But what did I miss out on? Here’s Cowen again:

Ideally foreign languages can be taught to individuals when they are young, well before high school, thus very much lowering the opportunity cost of such instruction.  Just toss out some of the other material, making sure to keep mathematics and English literacy.  Most of Western Europe does this quite well, and I hardly think of those children as miserable.  I don’t see why this has to cost anything at all.

Great point. Selfishly speaking, I hope to enroll my son in a dual-language school because I want my child to associate with the children of other parents who value multilingualism. In Canada the French Immersion system is often seen as a back-door extended education stream.

In my experience lots of kids drop out. I was one of two boys in my kindergarten class that made it to High School. There were probably 15-20 girls. I didn’t think of it this way at the time, but that’s all probably because the program was just really hard.

Education The Export of the Future? Enter the MOOC

Posted on by David Wright

The NYT had the first article on Coursera:

As part of a seismic shift in online learning that is reshaping higher education, Coursera, a year-old company founded by two Stanford University computer scientists, will announce on Tuesday that a dozen major research universities are joining the venture. In the fall, Coursera will offer 100 or more free massive open online courses, or MOOCs, that are expected to draw millions of students and adult learners globally.

And this one has the contract with its partner institutions.

The contract reveals that even Coursera isn’t yet sure how it will bring in revenue. A section at the end of the agreement, titled “Possible Company Monetization Strategies,” lists eight potential business models, including having companies sponsor courses. That means students taking a free course from Stanford University may eventually be barraged by banner ads or promotional messages. But the universities have the opportunity to veto any revenue-generating idea on a course-by-course basis, so very little is set in stone.

And this on the economics:

When and if money does come in, the universities will get 6 to 15 percent of the revenue, depending on how long they offer the course (and thus how long Coursera has to profit from it). The institutions will also get 20 percent of the gross profits, after accounting for costs and previous revenue paid. That means the company gets the vast majority of the cash flow.

I enrolled in the Database and Machine Learning courses in the precursor to Coursera last fall and I’m honestly still astonished it was all free.

I think they could definitely charge some small tuition fee and still attract literally millions of people for some of these classes. There’s a lot of money on the table there and a LOT of it is going to come from outside the US.

I am not confident that anything will seriously threaten the status economy of higher education. This might enhance it.

Imagine this model: online courses offered by MIT are 150% as difficult as the real thing because it teaches the same material with far less instruction. Class sizes can shrink and scholarships are offered to outstanding performers from the MOOC.

Suddenly it’s harder and cheaper to get into MIT.

Khan Academy Teaches Khan, Too

Posted on by David Wright

Here is an article attacking the Khan Academy (and a Khan response). Here is another, somewhat deeper critique. The upshot of both of these articles is the same: Khan isn’t a very good teacher.

What sour grapes! Not that this is the point, but find his videos quite good. Of course they aren’t perfect, but there is something extremely important going on here: Khan is a good teacher and he is getting better. Here are some quotes.

From a Khan employee:

Persistent misconception: “…we suggest that Khan Academy desperately needs voices of teaching experience. Khan could tap into any number of existing networks…”

Truth: We have four ex-teachers as full-time employees. We have two high school math teachers as consultants. One Harvard Doctoral candidate in Education and one post-doc in neuroscience at Stanford are in residence. One UPenn Professor is also likely to begin a sabbatical with us. We have a 3 person team dedicated to working with and getting feedback from our 50 pilot classrooms and the 15,000 teachers actively using KA in classrooms.

Persistent misconception: “…it certainly requires more than just “two minutes of research on Google,” which is how Khan describes his own pre-lesson routine.”

Truth: Go read Sal’s AMA response (includes the sentence “When I did organic chemistry, I spent 2 weeks immersing myself in the subject before making the first video”) before taking one of these “two minute” snipped quotes at face value: http://www.reddit.com/r/IAmA/comments/ntsco/i_am_salman_khan…. I’ve seen Sal’s face light up when he gets an unwieldy new shipment of textbooks to start studying in preparation for his videos. Does he dive right into some videos? Absolutely. Is claiming that his “pre-lesson routine” can always be dismissed as two minutes of Googling disingenuous and patently false? Absolutely.

From a mathematics education researcher:

That said, I have been up-front here on HN in suggesting ways that Khan Academy can improve, for example by building more online practice that is truly problems rather than exercises (379 days ago),

http://news.ycombinator.com/item?id=2760663

“Just for friendly advice to the Khan Academy exercise developers, I’ll repost my FAQ about the distinction between “exercises” and “problems” in mathematics education. It would be great to see more problems on the Khan Academy site.”

and the Khan Academy developers have been listening, and I have had interesting off-forum email interaction with them as they attempt to improve the instructional model at Khan Academy.

And here is the most important comment of all:

In general, I think mathematics is much too important a subject to be single-sourced from any source.

I’d add to that by saying that learning difficult things is really difficult. The best way of learning something is to learn it more than once and in more than one way. Khan is a free alternative way for people to learn math (or almost anything). This is incredibly valuable.

No teacher is going to be perfect but I’m sure that Khan is displaying above-average teaching competence in most of the subjects he chooses to teach. Khan will never be an outstanding teacher for his millions of students because there is something important in in-person instruction than cannot be replicated over video.

But very few students will have the opportunity to learn from more than a handful of outstanding teachers in their lives. For the majority of their experiences, Khan, and other instructors like him, will prove to be an immense help.

How many teachers are willing to expose their style for all to see and critique? Few, but those that do (look at this this article that suggets that teaching, like everything else humans do, is a skill) will become better at it. Good for us all.

What’s The Point of a Designation? (With a Venn Diagram!)

Posted on by David Wright

This is a fairly weak challenge to the pursuit of technical certification, but let’s start with it anyway:

I have come across a lot of my friends who aquired very nice percentage and received certificates though they have very minimal knowledge, or they have never worked on that particular technology. How crap is that, and now they are the proud owner of certificate, showcasing it proudly over their work desk. Ask them a very simple question on the technology that they have got certification, they would be for sure struggling to give answer to it. This is quite common, Certifications are being done only to get an extra point during their salary appraisals or job interviews. And I pity these corporate giants who would consider certification to be something remarkable.

How many out there would say, Yes certification is worth doing and it must be done to prove that you are good in a particular technology?

First point: if you’re an aspiring entrepreneur, stop right here. Stop reading, stop everything and go start building stuff. The longer you wait the harder it will be.

Second point: certifications are often required to work in a chosen field. That’s called occupational licensing and it’s often ridiculous. I’m going to concentrate on what real benefits certifications offer (in my experience) to the risk averse, hard working and ambitious. Doctors and lawyers need not read further.

I mentioned at the top the argument above is pretty easy to dismiss. Knocking something you haven’t actually tried? Striking down the straw man of heavily credentialed morons? Here’s a Venn diagram:

I did a fairly terrible job of scaling the image. The Red-only zone should be pretty tiny, the smallest of them all, really. Most people who spend the time and energy studying a topic do come away with some competence.

The deeper question is something anyone who spends a lot of time studying for tests should struggle with: is it worth your time? What does it get you?

Once upon a time I was at a dinner with a client who gave me some offhand advice: take some courses, they’ll teach you something, you’ll get an initialism for your business card and you’ll advance your career.

What terrible advice. I followed it, which I’ll get to later, but here’s what he should have said:

  1. Education is good, but remember its two functions: to teach you practical skills and signal your intelligence.
  2. Many certifications, unfortunately, are completely useless. By that I mean they teach you nothing useful and don’t signal a damn thing of any use to strangers.
  3. Most skills that will actually improve your job performance are best learned on the job. There is usually no good substitute for experience. Go help domain experts solve problems and effing pay attention. Study them.
  4. The most important skill of all? Check out this paper, which I’m actually reviewing for another blog post:

    In this study, measures of interpersonal and task-related skills were obtained from two groups of engineers: those nominated as “stars” by their managers and those nominated as “average”. Interestingly, the researchers found that the only distinguishing difference between the two was the stars’ interpersonal and affective skills. Specifically, the stars were better at developing rapport with coworkers and building extensive, loose networks of reliable problem solvers.

    Interpersonal skills. No certifications for that.

  5. Depressed yet? Well you should be. There is no reliable way to accelerate your career except to experience more. The only other possibility is to perhaps the change the trajectory of your career by changing the what kind of experience you get and how you respond to it.

NOW let’s talk about certifications.

Certifications work best as an introduction to a body of knowledge. Your goal should be enough understanding to follow a conversation between experts. Doesn’t sound like much, but this is incredibly important.

Imagine your mind dragging a net along behind it everywhere you go in life. You actually don’t have enough knowledge to properly interpret a lot of the experiences that pass through the net. Think of a certification as a way of shrinking the mesh of your net.

The second thing certifications can offer is the opportunity to work your ass off. Some certifications are really challenging to complete. Following my client’s advice I took a softball course, which has proven useless. The last module in it, however, was an introduction to finance which I really enjoyed (I was shocked – I HATED finance in undergrad and nearly failed it).

So next I tackled the CFA exams, then moved onto the CAS exams. Some would look at the amount of time I spent on these (over years and years) and shudder. Good. This makes them a fantastic signal of all kinds of qualities employers love.

But even the most grueling course yields nothing in isolation. What you want is the holy grail: high-value experience. By that I mean working with and learning from the best.

You see, the skills of the most incredibly skilled have afforded them prestige (always), wealth (often) and an extraordinary demand for their skills (always). They need help. And who are they going to pick? Putting nepotism to one side, they’re probably going to pick the highest status recruits, which means those with the strongest signals of quality.

I’m pretty fortunate to have a challenging certification available I can sink my teeth into. But programmers have an enormous expanse of open source projects they can attach their names to. And writers can always write, artists can, um, create, etc.

Certifications are ideal in mature industries where innovation is slow and the canon of skills relatively stable. In others, go online, the Internet has enabled quality signaling in just about any worthwhile pursuit.

But remember the iron law of education: if you don’t have to work hard for it then it probably isn’t worth your time.

Kids These Days (?)

Posted on by David Wright

Tyler Cowen sends us to this very interesting post by Steve Postrel on higher education:

My hypothesis is that it is precisely the dumbing down of U.S. education over the last decades that explains the increase in willingness to pay for education. The mechanism is diminishing marginal returns to education.

Typical graduate business school education has indeed become less rigorous over time, as has typical college education. But typical high school education has declined in quality just as much.

Postrel offers up four links to support the decline in rigor at all levels of education. Given that the entire argument rests on this point, we should follow those links:

1. Dumbing Down High School English. Opens up with this quote:

A recent ALSCW ( Association of Literary Scholars, Critics, and Writers) study finds that

a fragmented English curriculum and a neglect of close reading may explain why the reading skills of American high school students have shown little or no improvement in several decades despite substantial increases in funds for elementary and secondary education by federal and state governments.

After that it’s all hand waving. Disappointing because Postrel is worried about the LEVEL of education and this link despairs over the rate of positive change. Not relevant.

2. Link to this book. I haven’t read it. In this review, they say the book refers to one study in particular (pdf here) with this abstract:

Using multiple datasets from four different time periods, we document declines in academic time investment by full-time college students in the United States between 1961 and 2004. Full-time students allocated 40 hours per week toward class and studying in 1961, whereas by 2004 they were investing about 26 to 28 hours per week.

Students spend less time studying. I believe that.

But one thing the study does not control for is major type. Are engineers less well educated than in the past? I sincerely doubt that. Cowen may not find answers to declines in productivity improvements in this line of inquiry.

3. The Math Wars. Short version: the quality of math instruction in high schools has declined. I suppose I buy that. But I don’t envy my grandparents’ math instruction. I am sympathetic to reformers here.

4. On MBAs. Here’s a relevant quote on the author’s memory of his MBA student days and what he thinks has happened:

So, we read 30-40 academic journal articles per class. We became capable of digesting their content and, thereby, able to access new ideas 10-20 years ahead of widespread practice. We traced the trajectories of core research streams and, thus, came to recognize that subtle thinking is required of complex issues. We jammed into Merton Miller’s class, not because he was entertaining or capable of summarizing complex ideas into exquisite 10-bullet lists, but because everyone knew he was a genius and felt damn lucky to sit in his presence and glimpse into his thinking about finance. Excerpts from books by Tom Peters and other management “gurus” were not viewed as examples of special wisdom but, more accurately, of sloppy, shallow, unsubstantiated pap. That was a bad-ass education — one that served us well throughout our careers, not just in our next jobs.

What happened? Well, Business Week rankings coupled with the “Northwestern Innovation.” BW rated schools on: (1) student satisfaction, (2) recruiter assessment, and (3) research ranking. Northwestern, which was not a contender back then, realized that moving (2) or (3) could only happen veeery slooowly. Item (1), on the other hand, well, that could be manipulated almost instantaneously. And thus began the race to the bottom of the toilet. As far as I can tell, anything approaching the education I got has long since been abandoned.

More hand-waving, mostly, but I accept his premise. I’d point out a few things:

  • Chicago is a special place. Not every school has super-duper-star instructors and highly motivated students.
  • MBA programs are a ridiculously easy target for this kind of argument. I happen to think quite a lot of management ‘theory’ is complete garbage. Economics can be nearly as bad. Rigor in these subjects can often obscure learning outcomes. Let stories be told where stories must be told.

I find the idea that bad high school quality drives the increase in the college premium persuasive. I’m more sympathetic in respect of math than other subjects, though we must remain ever vigilant against cognitive bias. Complaining about the “kids these days” is usually total crap.