Diversity and Neg EMH

Those of you who know me personally will know how much I don’t like the word “diversity”, so for once let me give an argument in favor of it.

Efficient Market Hypothesis

One of the biggest take-aways I got from freshman year was something I like to call “neg EMH”, short for “negation of the Efficient Market Hypothesis”. This is a concept from economics which roughly says that the market self-regulates due to competition. For our purposes, we can of it more generally as saying

EMH: The world is big, and if you think you see something that everyone else misses you are probably wrong. If you are right, it doesn’t hurt to do a bit of due diligence anyway.

Its negation, then, would go something like

Neg EMH: The world isn’t that big; if you care, are thoughtful and intelligent, and have relevant expertise and skills, you shouldn’t be surprised to see something that no one else does, or do something that no one else can.

(These quotes are from a class at SPARC 2014.)

I think the biggest change I had freshman year was moving from the first point of view to the second point of view and even past it. It’s not merely possible that most people in the world are mistaken, it’s frequently the case. In fact, it’s frequently the case that most of the population is obviously mistaken. The first instance I saw of this was realizing how high school math education was (is) totally broken. Indeed there’s quite an elephant in the room:

You can’t actually be serious. Do people really think that knowing the Pythagorean Theorem will help in your daily life? I sure don’t, and I’m an aspiring mathematician . . . . It’s hilarious when you think about it. We’ve convinced millions of kids all over the country that they’re learning math because it’s useful in their lives, and they grudgingly believe it.

And ironically, the quote above is from an earlier blog post when I described how high school English was also broken. In both cases, people weren’t just wrong about something obscure, they were blatantly wrong about something which ought to have been patently obvious.

Icebergs

I realized that math was broken early in high school, and that writing was broken during about senior year, but at the time I dismissed it as just saying “high school is broken” and thought nothing more of it. It didn’t occur to me to think beyond that box.

Then I went to college and wasted a few hundred hours in one of the worst experiences of my life. It begin to dawn on me that perhaps there were more things that were broken than I thought. So I started looking harder, and I began to notice more and more inefficiencies. To give concrete examples:

  • In some (many) classes, people go to lectures to listen to a professor copy the notes he/she prepared onto a blackboard, then copy down the things on the blackboard into their notebooks. It’s almost like we never invented copy machines. [a]
  • Math textbooks are still written in a deadly boring, formal tone. To me, that’s just absurd. There are no jokes. There are few concrete examples. There is no “this is the standard example you should refer to”, “this is a surprising proof”, “this is just a routine calculation”. There’s rarely even “the main idea of the proof is…”. Instead, all you get is a sea of definitions, lemmas, theorems and proofs, which is often indecipherable.
    No one talks this way. No one thinks this way. Why are so many textbooks written this way?
  • Most people still use Microsoft Word (instead of Vim/LaTeX) and mouse-oriented operating systems and window managers (instead of tiling window managers, say). These are good examples of not buying O(n) returns at O(1) costs.

There are plenty more examples I have, but many are things I’m not really comfortable saying in public, so I’ll refrain from giving more examples [b].

Minority Rule

Now what does this have to do with diversity? Well, you might notice that most of the examples I gave had to do with math and college (and this becomes more true if you look at my full list). In fact, if you asked me what the two things I have the strongest feelings about are, I would say (i) math is usually taught poorly, especially at the lower levels, and (ii) college is an egregious waste of money [c].

What’s with that? Well, these happen to be the two things I think about the most.

This is a reflection of the totally obvious fact that if you spend lots of time thinking about something, you get to see things that most people don’t see [d]. I spend most of my time learning math, so I get to see when things are obviously broken [e]. I’d imagine someone who spends lot of time thinking about effective altruism (say) can probably see tons of inefficiencies there. And I had to spend lots of time thinking about the value of college because I’m applying for transfer. That’s why my views on the value of college are so strong.

This is the main value of talking to people with different specialties. It’s not merely that they see things differently; that’s tautologically true. The sinker is that smart people in different fields can often see that large portions of the population are blatantly wrong. It’s the word “blatantly” that’s important! I make fun of college all the time. I wonder who’s making fun of me.

A corollary: if most people believe X, but a smart “specialist” believes not X, the specialist is likely to be right, at least surprisingly often. (The “smart” condition cannot be dropped.) Put another way: an informed minority, perhaps even a single informed individual, is substantially better than an uninformed majority. That’s the power of neg EMH. So if I were looking for advice on picking colleges again, I’d first go find the kid who transferred out . . .

Go Do Good Things

Another corollary is that it’s easier to change the world than one might expect.

When I was younger, I used to think that “changing the world” was this glamorous thing that was near impossible, but that people attempted anyways for egotistic reasons; I explained to myself that stories of people succeeding were probably just survivor bias.

I still believe the survivor bias part, but I no longer think that most attempts are made by people trying to stroke their egos, but rather people who notice blatant market failures and feel compelled to act. In such a situation, it seems almost stupid not to take a shot. It’s not so much a feeling of “I can be the one to change the world” but rather “why on Earth has no one done this yet?”. That’s what I say all the time when I explain to people how I got the idea behind my geometry book, or any of my olympiad handouts for that matter. It is not Pride that changes the world, but Wrath.

Footnotes

[a] Of course, not all classes are like this, but many of them are, and I actively avoid classes which do this. As a rule of thumb, it’s easier to be personal in smaller classes, so taking higher-numbered classes over intro classes seems to be a good idea in general.

[b] Trust me, I would love to.

[c] Oops, that’s one of the things I was supposed to not say in public, although not one of the big ones. I’ll comment that I try very hard to only take classes that are worth my while. The fact that I actually learned math in high school is the only reason this is possible.

[d] Ironically, this sentence is an argument against individual diversity as follows: if you know a little bit about lots of things, then you don’t get the big “aha” moments in any of them.

[e] Not the math itself, of course, since in math we have proofs. (I imagine in other fields, you might notice things that are clearly wrong.) Though actually I’m told once you’ve delved enough into math research, you’ll can realize things that no one thought of before even though they should have. For example, Grothendieck on schemes…

Advertisements

6 thoughts on “Diversity and Neg EMH

  1. I agree with a lot of this post, although I think I also have some important disagreements.

    In terms of the EMH, it shouldn’t necessarily predict that you can’t find anything obviously bad, but rather that you can’t find anything that’s both obviously bad and trivially fixable (for instance I’m pretty sure other people have also said that high school is bad; but so far not much progress have been made to fix it). In this light, one should do due diligence not just on whether something that seems bad actually is, but also on what common failed approaches to fixing it have looked like.

    I am still very much in favor of doing both stages of due diligence described above, though I also believe that it is perhaps more possible than one would think to find opportunities where one has identified both an important problem and a credible approach to solving it.

    My overall model of the world is that it is not that hard to find important problems to work on; the bottleneck in most cases is finding thoughtful, talented individuals who are willing to devote considerable effort (~10,000 hours) to solving them. In this sense the EMH is false because the number of people who satisfy all 3 characteristics (thoughtful, talented, willing to devote considerable effort) is not that large. One of my hopes is that SPARC might lead to an increase in the number of such people.

    Finally, I disagree on the value of college (at least for places like Harvard / MIT). While there are some people (like, apparently, you) who do not get that much value out of it, there are other people that get extraordinary amounts of value out of it. I personally think that college was the best-spent 4 years of my life, due to a combination of the relationships that I developed and the opportunities I explored. This doesn’t mean that you are wrong to feel the way you do about college, but I think it does make it hard to claim that college is obviously a waste of time.

    Like

    • Thanks for writing. Also, I very much appreciate your post-script comment; I waffle a lot before posting anything non-math on this blog so it’s nice to hear that people do appreciate it :)

      > In terms of the EMH, it shouldn’t necessarily predict that you can’t find anything obviously bad, but rather that you can’t find anything that’s both obviously bad and trivially fixable
      Sure, that’s definitely true; I admit to abusing the economics terms to some extent. Certainly some of the things I mentioned that are obviously bad are that way because they are difficult to change.

      > In this sense the EMH is false because the number of people who satisfy all 3 characteristics (thoughtful, talented, willing to devote considerable effort) is not that large.
      OK, I’ll buy that; I’ve heard this sentiment expressed many times elsewhere, too.

      > Finally, I disagree on the value of college (at least for places like Harvard / MIT) . . . This doesn’t mean that you are wrong to feel the way you do about college, but I think it does make it hard to claim that college is obviously a waste of time.
      We should talk more about this at SPARC — I have some pretty extreme views on this that I probably shouldn’t say out loud.

      Like

  2. A counterpoint to the statement about individual diversity – one can argue that knowing a little about a lot of things allows someone to see the *connections* between two areas that a specialist in only one of the areas would miss.

    I suppose all that says about the world is that we need a diversity of both specialists *and* generalists, though.

    Like

  3. Pingback: Notes on Publishing My Textbook | Power Overwhelming

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s