Meritocracy is the worst form of admissions except for all the other ones

I’m now going to say something explicitly that I hinted at in June: I don’t think a student deserves to make MOP more because they had a higher score than another student.

I think it’s easy to get this impression because the selection for MOP is done by score cutoffs. So it sure looks that way.

But I don’t think MOP admissions (or contests in general) are meant to be a form of judgment. My primary agenda is to run a summer program that is good for its participants, and we get funding for N of them. For that, it’s not important which N students make it, as long as they are enthusiastic and adequately prepared. (Admittedly, for a camp like MOP, “adequately prepared” is a tall order). If anything, what I would hope to select for is the people who would get the most out of attending. This is correlated with, but not exactly the same as, score.

Two corollaries:

  • I support the requirement for full attendance at MOP. I know, it sucks for those star students who qualify for two conflicting and then have to choose. You have my apologies (and congratulations). But if you only come for 2 of 3 weeks, you took away a spot from someone who would have attended the whole time.
  • I am grateful to the European Girl’s MO for giving MOP an opportunity to balance the gender ratio somewhat; empirically, it seems to improve the camp atmosphere if the gender ratio is not 79:1.

Anyways, given my mixed feelings on meritocracy, I sometimes wonder whether MOP should do what every other summer camp does and have an application, or even a lottery. I think the answer is no, but I’m not sure. Some reasons I can think of behind using score only:

  1. MOP does have a (secondary) goal of IMO training, and as a result the program is almost insane in difficulty. For this reason you really do need students with significant existing background and ability. I think very few summer camps should explicitly have this level of achievement as a goal, even secondarily. But I think there should be at least one such camp, and it seems to be MOP.
  2. Selection by score is transparent and fair. There is little risk of favoritism, nepotism, etc. This matters a lot to me because, basically no matter how much I try to convince them otherwise, people will take any admissions decision as some sort of judgment, so better make it impersonal. (More cynically, I honestly think if MOP switched to a less transparent admissions process, we would be dealing with lawsuits within 15 years.)
  3. For better or worse, qualifying for MOP ends up being sort of a reward, so I want to set the incentives right and put the goalpost at “do maximally well on USAMO”. I think we design the USAMO well enough that preparation teaches you valuable lessons (math and otherwise). For an example of how not to set the goalpost, take most college admissions processes.

Honestly, the core issue might really be cultural, rather than an admissions problem. I wish there was a way we could do the MOP selection as we do now without also implicitly sending the (unintentional and undesirable) message that we value students based on how highly they scored.

An opening speech for MOP

While making preparations for this year’s MOP, I imagined to myself what I would say on orientation night if I was director of the camp, and came up with the following speech. I thought it might be nice to share on this blog. Of course, it represents my own views, not the actual views of MOP or MAA. And since I am not actually director of MOP, the speech was never given.

People sometimes ask me, why do we have international students at MOP? Doesn’t that mean we’re training teams from other countries? So I want to make this clear now: the purpose of MOP is not to train and select future IMO teams.

I know it might seem that way, because we invite by score and grade. But I really think the purpose of MOP is to give each one of you the experience of working hard and meeting new people, among other things. Learn math, face challenges, make friends, the usual good stuff, right? And that’s something you can get no matter what your final rank is, or whether you make IMO or EGMO or even next year’s MOP. The MOP community is an extended family, and you are all part of it now.

What I mean to say is, the camp is designed with all 80 of you in mind. It made me sad back in 2012 when one of my friends realized he had little chance of making it back next year, and told me that MAA shouldn’t have invited him to begin with. Even if I can only take six students to the IMO each year, I never forget the other 74 of you are part of MOP too.

This means one important thing: everyone who puts in their best shot deserves to be here. (And unfortunately this also means there are many other people who deserve to be here tonight too, and are not. Maybe they solved one or two fewer problems than you did; or maybe they even solved the same number of problems, but they are in 11th grade and you are in 10th grade.)

Therefore, I hope to see all of you put in your best effort. And I should say this is not easy to do, because MOP is brutal in many ways. The classes are mandatory, we have a 4.5-hour test every two days, and you will be constantly graded. You will likely miss problems that others claim are easy. You might find out you know less than you thought you did, and this can be discouraging. Especially in the last week, when we run the TSTST, many of you will suddenly realize just how strong Team USA is.

So I want to tell you now, stay determined in the face of adversity. This struggle is your own, and we promise it’s worth it, no matter the outcome. We are rooting for you, and your friends sitting around you are too. (And if the people around you aren’t your friends yet, change that asap.)

I switched to point-based problem sets

It’s not uncommon for technical books to include an admonition from the author that readers must do the exercises and problems. I always feel a little peculiar when I read such warnings. Will something bad happen to me if I don’t do the exercises and problems? Of course not. I’ll gain some time, but at the expense of depth of understanding. Sometimes that’s worth it. Sometimes it’s not.

— Michael Nielsen, Neural Networks and Deep Learning

1. Synopsis

I spent the first few days of my recent winter vacation transitioning all the problem sets for my students from a “traditional” format to a “point-based” format. Here’s a before and after.

Technical specification:

  • The traditional problem sets used to consist of a list of 6-9 olympiad problems of varying difficulty, for which you were expected to solve all problems over the course of two weeks.
  • The new point-based problem sets consist of 10-15 olympiad problems, each weighted either 2, 3, 5, or 9 points, and an explicit target goal for that problem set. There’s a spectrum of how many of the problems you need to solve depending on the topic and the version (I have multiple difficulty versions of many sets), but as a rough estimate the goal is maybe 60%-75% of the total possible points on the problem set. Usually, on each problem set there are 2-4 problems which I think are especially nice or important, and I signal this by coloring the problem weight in red.

In this post I want to talk a little bit about what motivated this change.

2. The old days

I guess for historical context I’ll start by talking about why I used to have a traditional format, although I’m mildly embarrassed at now, in hindsight.

When I first started out with designing my materials, I was actually basically always short on problems. Once you really get into designing olympiad materials, good problems begin to feel like tangible goods. Most problems I put on a handout are ones I’ve done personally, because otherwise, how are you supposed to know what the problem is like? This means I have to actually solve the problem, type up solution notes, and then decide how hard it is and what that problem teaches. This might take anywhere from 30 minutes to the entire afternoon, per problem. Now imagine you need 150 such problems to run a year’s curriculum, and you can see why the first year was so stressful. (I was very fortunate to have paid much of this cost in high school; I still remember many of the problems I did back as a student.)

So it seemed like a waste if I spent a lot of time vetting a problem and then my students didn’t do it, and as practical matter I didn’t have enough materials yet to have much leeway anyways. I told myself this would be fine: after all, if you couldn’t do a problem, all you had to do was tell me what you’ve tried, and then I’d walk you through the rest of it. So there’s no reason why you couldn’t finish the problem sets, right? (Ha. Ha. Ha.)

Now my problem bank has gotten much deeper, so I don’t have that excuse anymore. [1]

3. Agonizing over problem eight

But I’ll tell you now that even before I decided to switch to points, one of the biggest headaches was always whether to add in that an eighth problem that was really nice but also difficult. (When I first started teaching, my problem sets were typically seven problems long.) If you looked at the TeX source for some of my old handouts, you’ll see lots of problems commented out with a line saying “too long already”.

Teaching OTIS made me appreciate the amount of power I have on the other side of a mentor-student relationship. Basically, when I design a problem set, I am making decisions on behalf of the student: “these are the problems that I think you should work on”. Since my kids are all great students that respect me a lot, they will basically do whatever I tell them to.

That means I used to spend many hours agonizing over that eighth problem or whether to punt it. Yes, they’ll learn a lot if they solve (or don’t solve) it, but it will also take them another two or three hours on top of everything else they’re already doing (OTIS, school, trumpet, track, dance, social, blah blah blah). Is it worth those extra hours? Is it not? I’ve lost sleep over whether I made the right choice on the nights I ended up adding that last hard problem.

But in hindsight the right answer all along was to just let the students decide for themselves, because unlike your average high-school math teacher in a room of decked-out slackers, I have the best students in the world.

4. The morning I changed my mind

As I got a deeper database this year and commented more problems out, I started thinking about point-based problem sets. But I can tell you the exact moment when I decided to switch.

On the morning of Sunday November 5, I had a traditional problem set on my desk next to a point-based one. In both cases I had figured out how to do about half the problems required. I noticed that the way the half-full glass of water looked was quite different between them. In the first case, I was freaking out about the other half of the problems I hadn’t solved yet. In the second case, I was trying to decide which of the problems would be the most fun to do next.

Then I realized that OTIS was running on the traditional system, and what I had been doing to my students all semester! So instead of doing either problem set I began the first prototypes of the points system.

5. Count up

I’m worried I’ll get misinterpreted as arguing that students shouldn’t work hard. This is not really the point. If you read the specification at the beginning carefully, the number of problems the students are solving is actually roughly the same in both systems.

It might be more psychological than anything else: I want my kids to count how many problems they’ve solved, not how many problems they haven’t solved. Every problem you solve makes you better. Every problem you try and don’t solve makes you better, too. But a problem you didn’t have time to try doesn’t make you worse.

I’ll admit to being mildly pissed off at high school for having built this particular mindset into all my kids. The straight-A students sitting in calculus BC aren’t counting how many questions they’ve answered correctly when checking grades. They’re counting how many points they lost. The implicit message is that if you don’t do nearly all the questions, you’re a bad person because you didn’t try hard enough and you won’t learn anything this way and shame on you and…

That can’t possibly be correct. Imagine two calculus teachers A and B using the same textbook. Teacher A assigns 15 questions of homework a week, teacher B assigns 25 questions. All of teacher A’s students are failing by B’s standards. Fortunately, that’s not actually how the world works.

For this reason I’m glad that all the olympiad kids report their performance as “I solved problems 1,2,4,5” rather than “I missed problems 3,6”.

6. There are no stupid or lazy questions

The other wrong assumption I had about traditional problem sets was the bit about asking for help on problems you can’t solve. It turns out getting students to ask for help is a struggle. So one other hope is that with the point-based system is that if a student tries a problem, can’t solve it, and is too shy to ask, then they can switch to a different problem and read the solution later on. No need to get me involved with every single missed problem any more.

But anyways I have a hypothesis why asking for help seems so hard (though there are probably other reasons too).

You’ve all heard the teachers who remind students to always ask questions during lectures [2], because it means someone else has the same question. In other words: don’t be afraid to ask questions just because you’re afraid you’ll look dumb, because “there are no stupid questions“.

But I’ve rarely heard anyone say the same thing about problem sets.

As I’m writing this, I realize that this is actually the reason I’ve never been willing to go to office hours to ask my math professors for help on homework problems I’m stuck on. It’s not because I’m worried my professors will think I’m dumb. It’s because I’m worried they’ll think I didn’t try hard enough before I gave up and came to them for help, or even worse, that I just care about my grade. You’ve all heard the freshman biology TA’s complain about those kids that just come and ask them to check all their pset answers one by one, or that come to argue about points they got docked, or what-have-you. I didn’t want to be that guy.

Maybe this shaming is intentional if the class you’re teaching is full of slackers that don’t work unless you crack the whip. [3] But if you are teaching a math class that’s half MOPpers, I seriously don’t think we need guilt-trips for these kids whenever they can’t solve a USAMO3.

So for all my students, here’s my version of the message: there are no stupid questions, and there are no lazy questions.


  1. The other reason I used traditional problem sets at first was that I wanted to force the students to at least try the harder problems. This is actually my main remaining concern about switching to point-based problem sets: you could in principle always ignore the 9-point problems at the end. I tried to compensate for this by either marking some 9’s in red, or else making it difficult to reach the goal without solving at least one 9. I’m not sure this is enough.
  2. But if my question is “I zoned out for the last five minutes because I was responding to my friends on snapchat, what just happened?”, I don’t think most professors would take too kindly. So it’s not true literally all questions are welcome in lectures.
  3. As an example, the 3.091 class policies document includes FAQ such as “that sounds like a lot of work, is there a shortcut?”, “but what do I need to learn to pass the tests?”, and “but I just want to pass the tests…”. Also an entire paragraph explaining why skipping the final exam makes you a terrible person, including reasons such as “how do you anything is how you do everything”, “students earning A’s are invited to apply as tutors/graders”, and “in college it’s up to you to take responsibility for your academic career”, and so on ad nauseum.

Against the “Research vs. Olympiads” Mantra

There’s a Mantra that you often hear in math contest discussions: “math olympiads are very different from math research”. (For known instances, see O’Neil, Tao, and more. More neutral stances: Monks, Xu.)

It’s true. And I wish people would stop saying it.

Every time I’ve heard the Mantra, it set off a little red siren in my head: something felt wrong. And I could never figure out quite why until last July. There was some (silly) forum discussion about how Allen Liu had done extraordinarily on math contests over the past year. Then someone says:

A: Darn, what math problem can he not do?!

B: I’ll go out on a limb and say that the answer to this is “most of the problems worth asking.” We’ll see where this stands in two years, at which point the answer will almost certainly change, but research \neq Olympiads.

Then it hit me.

Ping-pong vs. Tennis

Let’s try the following thought experiment. Consider a world-class ping-pong player, call her Sarah. She has a fan-base talking about her pr0 ping-pong skills. Then someone comes along as says:

Well, table tennis isn’t the same as tennis.

To which I and everyone else reasonable would say, “uh, so what?”. It’s true, but totally irrelevant; ping-pong and tennis are just not related. Maybe Sarah will be better than average at tennis, but there’s no reason to expect her to be world-class in that too.

And yet we say exactly the same thing for olympiads versus research. Someone wins the IMO, out pops the Mantra. Even if the Mantra is true when taken literally, it’s implicitly sending the message there’s something wrong with being good at contests and not good at research.

So now I ask: just what is wrong with that? To answer this question, I first need to answer: “what is math?”.

There’s been a trick played with this debate, and you can’t see it unless you taboo the word “math”. The word “math” can refer to a bunch of things, like:

  • Training for contest problems like USAMO/IMO, or
  • Learning undergraduate/graduate materials like algebra and analysis, or
  • Working on open problems and conjectures (“research”).

So here’s the trick. The research community managed to claim the name “math”, leaving only “math contests” for the olympiad community. Now the sentence

“Math contests should be relevant to math”

seems totally innocuous. But taboo the world “math”, and you get

“Olympiads should be relevant to research”

and then you notice something’s wrong. In other words, since “math” is a substring of “math contests”, it suddenly seems like the olympiads are subordinate to research. All because of an accident in naming.

Since when? Everyone agrees that olympiads and research are different things, but it does not then follow that “olympiads are useless”. Even if ping-pong is called “table tennis”, that doesn’t mean the top ping-pong players are somehow inferior to top tennis players. (And the scary thing is that in a world without the name “ping-pong”, I can imagine some people actually thinking so.)

I think for many students, olympiads do a lot of good, independent of any value to future math research. Math olympiads give high school students something interesting to work on, and even the training process for a contest such that the IMO carries valuable life lessons: it teaches you how to work hard even in the face of possible failure, and what it’s like to be competitive at an international level (i.e. what it’s like to become really good at something after years of hard work). The peer group that math contests give is also wonderful, and quite similar to the kind of people you’d meet at a top-tier university (and in some cases, they’re more or less the same people). And the problem solving ability you gain from math contests is indisputably helpful elsewhere in life. Consequently, I’m well on record as saying the biggest benefits of math contests have nothing to do with math.

There are also more mundane (but valid) reasons (they help get students out of the classroom, and other standard blurbs about STEM and so on). And as a matter of taste I also think contest problems are interesting and beautiful in their own right. You could even try to make more direct comparisons (for example, I’d guess the average arXiv paper in algebraic geometry gets less attention than the average IMO geometry problem), but that’s a point for another blog post entirely.

The Right and Virtuous Path

Which now leads me to what I think is a culture issue.

MOP alumni prior to maybe 2010 or so were classified into two groups. They would either go on to math research, which was somehow seen as the “right and virtuous path“, or they would defect to software/finance/applied math/etc. Somehow there is always this implicit, unspoken message that the smart MOPpers do math research and the dumb MOPpers drop out.

I’ll tell you how I realized why I didn’t like the Mantra: it’s because the only time I hear the Mantra is when someone is belittling olympiad medalists.

The Mantra says that the USA winning the IMO is no big deal. The Mantra says Allen Liu isn’t part of the “smart club” until he succeeds in research too. The Mantra says that the countless time and energy put into running each year’s MOP are a waste of time. The Mantra says that the students who eventually drop out of math research are “not actually good at math” and “just good at taking tests”. The Mantra even tells outsiders that they, too, can be great researchers, because olympiads are useless anyways.

The Mantra is math research’s recruiting slogan.

And I think this is harmful. The purpose of olympiads was never to produce more math researchers. If it’s really the case that olympiads and research are totally different, then we should expect relatively few olympiad students to go into research; yet in practice, a lot of them do. I think one could make a case that a lot of the past olympiad students are going into math research without realizing that they’re getting into something totally unrelated, just because the sign at the door said “math”. One could also make a case that it’s very harmful for those that don’t do research, or try research and then decide they don’t like it: suddenly these students don’t think they’re “good at math” any more, they’re not smart enough be a mathematician, etc.

But we need this kind of problem-solving skill and talent too much for it to all be spent on computing R(6,6). Richard Rusczyk’s take from Math Prize for Girls 2014 is:

When people ask me, am I disappointed when my students don’t go off and be mathematicians, my answer is I’d be very disappointed if they all did. We need people who can think about these complex problems and solve really hard problems they haven’t seen before everywhere. It’s not just in math, it’s not just in the sciences, it’s not just in medicine — I mean, what we’d give to get some of them in Congress!

Academia is a fine career, but there’s tons of other options out there: the research community may denounce those who switch out as failures, but I’m sure society will take them with open arms.

To close, I really like this (sarcastic) comment from Steven Karp (near bottom):

Contest math is inaccessible to over 90% of people as it is, and then we’re supposed to tell those that get it that even that isn’t real math? While we’re at it, let’s tell Vi Hart to stop making videos because they don’t accurately represent math research.

Addendums (response to comments)

Thanks first of all for the many long and thoughtful comments from everyone (both here, on Facebook, in private, and so on). It’s given me a lot to think about.

Here’s my responses to some of the points that were raised, which is necessarily incomplete because of the volume of discussion.

  1. To start off, it was suggested I should explicitly clarify: I do not mean to imply that people who didn’t do well on contests cannot do well in math research. So let me say that now.

  2. My favorite comment that I got was that in fact this whole post pattern matches with bravery debates.

    On one hand you have lots of olympiad students who actually FEEL BAD about winning medals because they “weren’t doing real math”. But on the other hand there are students whose parents tell them to not pursue math as a major or career because of low contest scores. These students (and their parents) would benefit a lot from the Mantra; so I concede that there are indeed good use cases of the Mantra (such as those that Anonymous Chicken, betaveros describe below) and in particular the Mantra is not intrinsically bad.

    Which of these use is the “common use” probably depends on which tribes you are part of (guess which one I see more?). It’s interesting in that in this case, the two sides actually agree on the basic fact (that contests and research are not so correlated).

  3. Some people point out that research is a career while contests aren’t. I am not convinced by this; I don’t think “is a career” is a good metric for measuring value to society, and can think of several examples of actual jobs that I think really should not exist (not saying any names). In addition, I think that if the general public understood what mathematicians actually do for a career, they just might be a little less willing to pay us.

    I think there’s an interesting discussion about whether contests / research are “valuable” or not, but I don’t think the answer is one-sided; this would warrant a whole different debate (and would derail the entire post if I tried to address it).

  4. Some people point out that training for olympiads yields diminishing returns (e.g. learning Muirhead and Schur is probably not useful for anything else). I guess this is true, but isn’t it true of almost anything? Maybe the point is supposed to be “olympiads aren’t everything”, which is agreeable (see below).

  5. The other favorite comment I got was from Another Chicken, who points out below that the olympiad tribe itself is elitist: they tend to wall themselves off from outsiders (I certainly do this), and undervalue anything that isn’t hard technical problems.

    I concede these are real problems with the olympiad community. Again, this could be a whole different blog post.

    But I think this comment missed the point of this post. It is probably fine (albeit patronizing) to encourage olympiad students to expand; but I have a big problem with framing it as “spend time on not-contests because research“. That’s the real issue with the Mantra: it is often used as a recruitment slogan, telling students that research is the next true test after the IMO has been conquered.

    Changing the Golden Metric from olympiads to research seems to just make the world more egotistic than it already is.

___ Students Have to Suffer

This will be old news to most of the readership of this blog, but I realize I’ve never written it down, so time to fix that.

Fill in the blank

Let’s begin by playing a game of “fill in the blank”. Suppose that today, the director of secondary education at your high school says:

“___ students just have to suffer.”

This is not a pleasant sentence. Fill in that blank with a gender, and you’d be fired tomorrow morning. Fill in that blank with an ethnic group, and you’d be fired in an hour. Fill in that blank with “special needs”, and you’d be be sued. Heck, forget “___ students”, replace that with “You”. Can you see someone’s career flashing before their eyes? How could you possibly get away with saying that about any group of students?

Those 500 hours

“Smart students just have to suffer.”
Director of Secondary Education at Fremont Unified School District

This happened to me. I haven’t told this story enough, so I will tell it some more.

When I was a senior in high school, I was enrolled in two classes and would thereafter run off to take graduate math at UC Berkeley. (Notes here.) This was fantastic and worked for a few weeks, so I got to learn real analysis and algebraic combinatorics from some nice professors.

Then the school district found out, and called me in for a meeting. The big guy shows up, and gives me this golden quote. I was then required to enroll in five classes a day, the minimum number of classes required for me to count towards the average daily attendance funding for my school district.

And that is why, for three periods a day, five days a week, I was forced to sit in the front office, saying “Hi, how may I help you?”.

(I didn’t even get paid! Could’ve asked for a cut of that ADA funding. It didn’t all go to waste though; I spent the time writing a book.)

Everywhere Else

Since I’ve had fun picking on my school district, I will now pick on the Department of Education.

“While challenging and improving the mathematical problem-solving skills of high-performing students are surely every-day objectives of those who teach such students, it is not a problem, relatively speaking, of major import in American education.”
Department of Education Reviewer

Oh boy.

The point is that the problem of neglecting gifted students isn’t at the level of individual teachers. It’s not a problem at the level of individual schools, or individual cities. This is a problem with national culture. The problem is that as a culture we think it’s okay to say a sentence like that.

Replace “high-performing” with any adjective you want. Any gender, any social class, any ethnic group, whatever, and you will get a backlash. But we’ve decided that it’s okay to mistreat the gifted students, because no one complains at that.

Maybe it’s too much to ask that schools do something special for top students. Can you at least not get in their way? Like not forcing students to be an office assistant for 500 hours to obtain ADA funding? Or more generally, how about just not forcing students to take classes which are clearly a waste of time for them?

Next Actions

So what can you do to change the national culture? As far as I can tell, this is mostly a lost clause. I wouldn’t bother trying.

The reason I wrote this post because I went through most of high school not really being aware of just how badly I was being mistreated. I’m really writing this for myself four years ago to point out that, man, us nerds really got the ugly end of the deal.

What you can do (and should) is make small local changes. You can persuade individual schools to make exceptions for a kid, and frequently individual teachers will do what they can to help a gifted student as well. Each individual student has good chance of finding a way around the big bureaucrats that rule the wastelands.

Ask a lot of people: if one administrator says no, ignore them and ask another one. Be prepared to hear “no” a lot, but keep waiting for the one or two crucial “yes” moments. If push comes to shove, switch schools, apply to college early, etc. Take the effort to get this one right. (See 56:30-60:00 of this for more on that.)

Dear past self, yell a little harder at the big guy when he comes, maybe you can save yourself 500 hours as an office aid.

Addendum: A Happy Story

In the comments, someone wrote the following:

Did your mistreatment as a gifted student hinder you in any significant way? … Where would you be today had the system not failed you?

I think it’s impossible to know. But here’s another story.

  • When I was in 7th grade, my school tried to force me to take pre-algebra. My mom begged the school teachers until they finally relented and let me take Algebra I. At the time, my 12-year-old self couldn’t have cared less: both classes were too easy for me, and I spent most of Algebra I playing Tetris on my TI-89.
  • Two years later this happened again: the school wanted to force me to take Algebra II. This time, my mom begged the teachers to let me take precalculus instead, which they eventually did. My 14-year-old self also couldn’t care less; both classes were too easy anyways, and I spent most of precalculus playing osu on my iPhone.
  • Two years later I was in Calculus BC, again bored to tears and in the last HS math class offered. That’s when my parents were able to persuade the school to let me take classes at UC Berkeley instead, since I had exhausted the HS math curriculum. I did very well in my first undergraduate classes, which then allowed me to take graduate classes for the rest of high school.

These professors were the ones that wrote my reference letters for college applications, which got me into all the top schools in the country (Berkeley, UCLA, MIT, Princeton, Stanford, Harvard). Without these reference letters I would certainly not have had as many options; winning the USAMO and making the IMO didn’t happen for me until the end of senior year.

But it wasn’t until I met the guy quoted above that I found out that I had unwittingly “broken district rules”, and technically shouldn’t have happened. (Belated thanks to those individuals who stuck out their necks for me!)

So here’s a surprisingly clear example of a near miss. Suppose that my mom had been more polite, or my school had been a little more firm, and any of the three events above didn’t occur. Not only would I have lost some college choices (potentially including MIT), I wouldn’t even know that this was the key event I could have changed.

[Bonus question: I estimate about 2% of USA high school students take the AMC. How would my life have changed if I had been in the other 98%?]

By analogy, if you ask me now what ways I’ve been affected, how am I to tell you? Without an Earth simulator I can’t point to which of the other 100 times I was mistreated hurt me the most. All I can do is point out that I (and many others) are being mistreated, which really should not be okay in the first place.

Or maybe a better argument: if you halve the amount of time I have to learn by making me go to high school for six hours a day, and then I get a gold medal at the IMO, I do not think the conclusion should be “it had no effect”, but something more like “I could have gotten two gold medals at the IMO”. Not that gold medals scale linearly with time but you get the point.

Against Hook-Length on USAMO 2016/2

A recent USAMO problem asked the contestant to prove that

\displaystyle  (k^2)! \cdot \prod_{j=0}^{k-1} \frac{j!}{(j+k)!}

is an integer for every {k \in \mathbb N}. Unfortunately, it appears that this is a special case of the so-called hook-length formula, applied to a {k \times k} Young tableau, and several students appealed to this fact without proof to produce one-line solutions lacking any substance. This has led to a major controversy about how such solutions should be graded, in particular whether they should receive the {7^-} treatment for “essentially correct solutions”, or the {0^+} treatment for “essentially not solved”.

In this post I want to argue that I think that these solutions deserve a score of {1}.

1. Disclaimers

However, before I do so, I would like to make some disclaimers:

  • This issue is apparently extremely polarized: everyone seems to strongly believe one side or the other.
  • This was an extremely poor choice of a USAMO problem, and so there is no “good” way to grade the HL solutions, only a least bad way. The correct solution to the dilemma is to not have used the problem at all. Yet here’s the bloodied patient, and here we are in the emergency room.
  • While I am a grader for the USAMO, I am one of many graders, and what I say in this post does not necessarily reflect the viewpoints of other USAMO graders or what score the problem will actually receive. In other words, this is my own view and not official in any way.

One last remark is that I do not consider the hook-length formula to be a “well-known” result like so many contestants seem to want to pretend it is. However, this results in the danger of what constitutes “well-known” or not. So in what follows I’ll pretend that the HL formula is about as well-known as, say, the Pascal or Zsigmondy theorem, even though I personally don’t think that this is the case.

One final disclaimer: I am unlikely to respond to further comments about an issue this polarized, since I have already spent many hours doing so, and I’ve done enough of my duty. So if I don’t respond to a comment of yours, please don’t take it personally.

2. Rule for citations

Here is the policy I use for citations when grading:

  • You can cite any named result as long as it does not trivialize the problem.
  • If the result trivializes the problem, then you are required to prove the result (or otherwise demonstrate you understand the proof) in order to use it.

This is what I’ve heard every time I have asked or answered this question; I have never heard anything to the contrary.

Some people apparently want to nit-pick about how “trivialize” is not objective. I think this is silly. If you really want a definition of “trivialize”, you can take “equivalent to the problem or a generalization of the problem” as a rule of thumb.

Clearly it follows from my rule above that the hook-length formula deserves {0^+} grading, so the remainder of the post is dedicated to justifying why I think this is the correct rule.

3. Subjective grading

I would rather have an accurate subjective criteria than a poor objective one.

In an ideal world, grading would be completely objective: a solution which solves the problem earns {7^-} points and a solution which does not solve the problem earns {0^+} points. But in practice this is of course not possible, unless we expect our contestants to write their solutions in a formal language like Coq. Since this is totally infeasible, we instead use informal proofs: students write their solutions in English and human graders assign a score based on whether the solution could in principle be compiled into a formal language proof.

What this means is that in grading, there are subjective decisions made all of the time. A good example of this is omitting steps. Suppose a student writes “case {B} is similar [to case {A}]”. Then the grader has to decide whether or not the student actually knows that the other case is similar or not, or is just bluffing. One one extreme, if {A} and {B} really are identical, then the grader would probably accept the claim. On the other extreme, if {A} and {B} have some substantial differences, then the grader would almost certainly reject the claim. This implies that there is some intermediate “gray area” which cannot be objectively defined.

Fortunately, most of these decisions are clear, thus USAMO grading appears externally to be mostly objective. In other words, accurate grading and objective grading are correlated (yay math!), but they are not exactly the same. All scores are ultimately discretion by human graders, and not some set of rigid guidelines.

Citations are a special case of this. By citing a theorem, a student is taking advantage of convention that a well-known proof to both the student and grader can be omitted from the write-up.

4. Citing the problem

In light of this I don’t think you should get points for citing the problem. I think we all agree that if a student writes “this is a special case of IMO Shortlist 1999 G8”, they shouldn’t get very many points.

The issue with citing HL in lieu of solving the problem is that the hook-length formula is very hard to prove, and it is not reasonable to do so in an olympiad setting. Consequently, it is near certain that these students have essentially zero understanding of the solution to the problem; they have not solved it, they have only named it.

Similarly, I think you should not earn points for trivializing a problem using Dirichlet, Prime Number Theorem, Zsigmondy, etc. This is also historically consistent with the way that grading has been done in the past (hi Palmer!).

5. Citing intermediate steps

Now consider the usage of difficult theorems such as Dirichlet, Prime Number Theorem, etc. on a solution in which they are merely an intermediate step rather than the entire problem. The consensus here is that citing these results is okay (though it is not unanimous; some super harsh graders also want to take off points here, but they are very few in my experience).

I think this is acceptable, because in this case the contestant has done 90% of the solution, and cannot do the remaining 10% but recognize it as well-known. In my eyes this is considered as essentially solving the problem, because the last missing bit is a standard fact. But somehow if you cannot do 100% of the problem, I don’t think that counts as solving the problem.

What I mean is there is a subjective dependence both on how much of the solution the student actually understands, and how accessible the result is. Unfortunately, the HL solutions earn the worst possible rank in both of the categories: the student understands 0% of the solution and moreover the result is completely inaccessible.

6. Common complaints

Here are the various complaints that people have made to me.

  • “HL is well-known.”
    Well, I don’t think it is, but in any case that’s not the point; you cannot cite a result which is (a) more general than the problem, and (b) to which you don’t understand the proof.

  • “Your criteria is subjective!”
    So what? I would rather have an accurate subjective criteria than a poor objective one.

  • “It’s the problem writer’s fault, so students should get {7}.”
    This is not an ethics issue. It is not appropriate to award points of sympathy on a serious contest such as the USAMO; score inflation is not going to help anyone.

  • “It’s elitist for the graders to decide what counts as trivialized.”
    That’s the grader’s job. Again I would rather have an accurate subjective criteria than a poor objective one. In practice I think the graders are very reasonable about decisions.

  • “I don’t think anyone disputes that the HL solution is a correct one, so certainly the math dictates a {7^-}.”
    I dispute it: I don’t think citing HL is a solution at all.

  • “Why do we let students use Pascal / Cauchy / etc?”
    Because these results are much more reasonable to prove, and the “one-line” solutions using Pascal and Cauchy are not completely trivial; the length of a solution is not the same thing as its difficulty. Of course this is ultimately subjective, but I would rather have an accurate subjective criteria than a poor objective one.

  • “HL solutions have substance, because we made an observation that the quantity is actually the number of ways to do something.”
    That’s why I wish to award {1} instead of {0}.

  • “Your rule isn’t written anywhere.”
    Unfortunately, none of the rules governing USAMO are written anywhere. I agree this is bad and the USAMO should publish some more clear description of the grading mechanics.

  • “The proof of the HLF isn’t even that complicated.”
    Are you joking me?

In summary, I don’t think it is appropriate to give full marks to a student who cannot solve a problem just because they can name it.

Things SPARC

[EDIT 2018/03/05: This description seems significantly less accurate to me now than it did a few years ago, both because my views/values have changed substantially, and because SPARC has changed direction substantially since I attended as a junior counselor in 2015. I’ll leave it here as a reference, but should be taken with a grain of salt.]

I often get asked about what I learned from the SPARC summer camp. This is hard to describe and I never manage to give as a good of an answer as I want, so I want to take the time to write down something concrete now. For context: I attended SPARC in 2013 and 2014 and again as a counselor in 2015, so this post is long overdue (but better late than never).

(For those of you still in high school: applications for 2016 are now open, due March 1, 2016. The program is completely free including room/board and you don’t need rec letters, so there is no reason to not apply.)

The short version is that maybe 1/4 of the life skills I use on a regular basis are things I picked up from SPARC. (The rest came from some combination of math contests and living in college dorms.) On paper SPARC seems like a math or CS camp, but there is a strong emphasis on practicality in the sense that the instructors specifically want to teach you things that you can apply in life. So in addition to technical classes on Bayes’ theorem and the like, you’ll have classes on much “softer” topics like

  • Posture (literally about having good body posture)
  • Aversion factoring (e.g. understanding why I’m not exercising and fixing it)
  • Expanding comfort zones (with hands-on practice; my year I learned to climb trees)

and so on. This makes it hard to compare to other math camps like MOP (though if you insist on drawing a comparison, I think many MOP+SPARC students agree they learned more from SPARC).

Some more testimonials:

Now, here is my own list of concrete things which SPARC has taught me, in no particular order:

  • Being significantly more introspective / reflective about life. Example: realizing that some class/activity/etc. are not adding much value to life and dropping them.
  • Being interested in optimizing life in general; I now find it fun to think about how to be more productive and live life well the same way I like to think about hard math problems.
  • Thinking about thinking: things like mental models, cognitive biases, emotions, aversions, System 1 vs System 2,
  • Becoming very aggressive at conserving time. I’m much more willing to trade money for time, and actively asking whether I really need to do something, or if I can just axe it.
  • Using game theory concepts to think about the world. College tuition is expensive because this is the Nash equilibrium. Recognizing real-life situations which are well understood as games, like prisoner’s dilemma, chicken, etc.
  • Applying Bayes’ theorem and expected value to real life. Trying out X activity has constant cost but potentially large payoffs, hence large positive EV.
  • Being able to use probabilities in a meaningful way. Being able to tell the difference between being 90% confident and 70% confident in an event happening.
  • Actively buying O(n) returns for O(1) cost.
  • Being more willing to take less conventional paths, like essentially dropping out of high school to train for the IMO (I describe this in the first FAQ here). Another good example I haven’t done myself (yet) is taking a gap year.
  • Using Workflowy. It’s a big part of why I can think as clearly as I do. In context of SPARC, this is a special case of understanding the idea of working memory, which is also an idea I picked up from camp.
  • Writing a lot more. This is probably actually a consequence of the things above rather than something that I directly learned from SPARC; some combination of understanding working memory well, and being much more reflective.
  • Peer group and culture. This is a bigger one than people realize. In the same way that math contests establish a group of people where it’s cool to think about hard math problems, the SPARC network establishes a group of people with a culture of encouraging people to think about rationality. It’s very hard to be good at reflection in an isolated environment! SPARC lets you see how other people go about thinking about how to live life well and gives you other people to bounce ideas off of.

I’m sure there’s other things, but it’s hard for me to notice since it’s been so long since I had to live life pre-SPARC. And there’s some things that other people learned from SPARC that never stuck with me (lots of the social skills, for example). Much like your first time attending MOP, there will be more things to learn than you’ll actually be able to absorb. So the list above is only the things that I myself learned, and in fact I think the set of things you acquire from SPARC more or less molds to whichever particular things matter to you most.

Time is Money

At some point since arriving at college, when I was trying to decide whether something was worth the time, I started quantifying it in term of money (examples follow). This worked pretty well for me: money is easy to quantify, and the thought of spending money is something I’m naturally averse to.

The most noticeable effect was that it made me a lot more conscious of wasted time.

Roughly, I estimate that my time is worth about $30 an hour to me — I can get a $20/hour job without too much trouble, but I imagine that there would be things I’d rather do than a $20/hour job. And time is worth strictly more than the earned money — time cannot be saved, stored, or earned back.

With this figure, I got some pretty interesting translations:

  • “I was on Facebook for two hours today” is “I paid $60 to read my Facebook news feed”.
  • “I slept in until 1PM today” is “I paid $150 to avoid getting out of bed” (assuming one can wake up at 8AM and still function).
  • “I spent all of today procrastinating on work” is “I just wasted $300” (10 waking hours).

Things on the left side are things I overhear all the time at lunch in the freshman dining hall. I’ll bet the right side is something that no one would feel comfortable saying. We’re so used to spending time we forget how valuable it is!

And people think I’m weird for waking up early. Who wouldn’t wake up an extra couple hours early if they got paid $60 to do so? Or more negatively: would you still press snooze if it cost $60?

It’s even more fun to take this and scale up.

  • “I didn’t drop this class even though I didn’t like it” is “I passed up a refund for a $5,000 item I didn’t like”.
  • “I’m not sure what I got out of my last year at college” is “I bought a $100,000 item but I’m not sure what it was”.

Money is remarkably good at putting things in perspective.


I’ve recently come to believe that “deep conversations” are overrated. Here is why.


Human short term memory is pretty crummy. Here is an illustration from linguistics:

A man that a woman that a child that a bird that I heard saw knows loves

This is a well-formed English phrase. And yet parsing it is difficult, because you need a stack of size four. Four is a pretty big number.

And that’s after I’ve written the sentence down for you, so your eyes could scan it two or three times to try and parse it. Imagine if I instead said this sentence aloud.

Other examples include any object with some moderately complex structure:

Let ABC be a triangle and let AD, BE, CF be altitudes concurrent at the orthocenter H.

This is not a very complicated diagram, but it’s also very difficult to capture in your head unless you’ve seen it before — and again, that’s after I’ve written it down for your viewing pleasure.

Now imagine what you’re talking about isn’t just six lines and seven points, but “what do you think the point of college is?”, or “should a high school diploma be required to obtain a driver’s license?”, or “what is algebraic geometry about?” (all examples from my life, mind you). The answer to these questions is far more complex than the trivial examples I’ve given above. To try and talk about these things merely by voice seems fruitless.

It’s worth pointing out that you can get away with things that have a lot of breadth as long they do not have depth — loosely, as long as there are not too many dependencies. To give an example, the linguistics example is tricky because all the subjects and verbs depend on each other. The geometry diagram is tricky because the points are all tied together in a certain way. But I could read the first chapter of And Then I Thought I Was A Fish out loud, or tell you the story of the cute girl I met three summers ago, because the parts don’t depend (as much) on one another: at any point in a story you can remember the last couple sentences and still enjoy the story. But if I try to read you the first chapter of The Rising Sea: Foundations of Algebraic Geometry, it would make a good bedtime story only because you’d probably fall asleep.

It just strikes me as bizarre that people talk about “deep” issues without writing a single thing down. I think if you’re having lunch with a friend and discussing something like this, you ought to at least have a piece of paper out on the table where you can both jot down the main ideas of what’s been said. It doesn’t need to have every word because then you just get bloat, but still, at least get the key insights somewhere visible. (That’s what presentation slides and blackboards are for, right?)


The other strange thing is that in conversations, you have to process and respond in real-time. You can only spend as long thinking about a sentence as it takes for the next one to be said.

This is fine if I ask you a question such as “what is your birthday?”, because lookup queries are fast. It’s fine if I ask you “what did you think of X book you read?”, again because it is just a lookup query. Note that this is true even if you spent a long time reading and thinking about the book, because the computation was already done. It’s even fine if I ask “what is two plus five?” because it takes not very long to add.

But if I ask “what do you think about the war on drugs?” and you haven’t been thinking much about it, then the best answer you can give is “I don’t know”; because you can’t do a lookup query for an answer you haven’t computed yet.

Put another way, suppose someone asks me some complex question like “how do I get better at math contests?”, and I respond “do a lot of problems that are right above your ability”. One of two possible things just happened:

  1. I had already thought about this question, and this is a pre-computed answer, or
  2. I came up with this in the half second between the end of your question and the start of my response. (Though you can increase this time by pre-pending “uh”, “like”, “I think”.)

In other words, the fast nature of conversations prevents anything other than cache lookups and first impressions (or I suppose possibly a combination of both). And if the issue you’re talking about is sufficiently complex, first impressions are likely not so insightful. So if I sound really smart in a conversation, the only reason is that you’re asking questions I already pre-computed good answers for.

In other words, the best you can do from a typical conversation is learn what people’s existing ideas are. There isn’t a tractable way to generate new ideas from feedback, just because the time-scale involved is too small. Eliezer Yudkowsky makes a similar point in a Less Wrong Post:

If you want to sound deep, you can never say anything that is more than a single step of inferential distance away from your listener’s current mental state. That’s just the way it is.


I will now point out that both issues I mentioned have easy partial fixes. If I’m correct, then deep conversations can be substantially enhanced if we use paper or blackboard or anything else, and agree it is socially acceptable to take a minute to respond. Neither of these actions will completely alleviate their respective problems, but trust me when I say having 60 seconds to think is a world of difference compared to 2.

Both of these initially struck me as weird conclusions, but they do seem to make sense on closer inspection. In fact, I have actually seen both done in practice (albeit not simultaneously). So this means I have a way to test what I’ve written in this post now…


Apparently even people on Quora want to know why I transferred from Harvard to MIT. Since I’ve been asked this question way too many times, I guess I should give an answer, once and for all.

There were plenty of reasons (and anti-reasons). I should say some anti-reasons first to give due credit — the Harvard math department is fantastic, and Harvard gives you significantly more freedom than MIT to take whatever you want. These were the main reasons why transferring was a difficult decision, and in fact I’m only ~70% sure I might the right choice.

Ultimately, the main reason I transferred was due to the housing.

At MIT, you basically get to choose where you live. All the dorms, and even floors within dorms, are different: living on 3rd West versus living on 5th East might as well be going to different colleges. Even if for some bizarre reason you hate 90% of the students at MIT you can still have a fantastic social experience if you’re in a dorm you like.

This is not true at Harvard, which shoves you in dorms more or less at random. Specifically,

  • In freshman year, you are assigned a random dorm, and eat in a segregated dining hall (Annenberg) exclusively with freshman. All students are placed on a mandatory unlimited meal plan, I guess to discourage them from eating out.
  • After freshman year, you get a random House, and eat in a dining hall built into the House. There are restrictions that make it deliberately difficult to eat at other Houses.

The result of this random mixing is that (a) you only know people in your own year, and (b) zero dorm culture. Lounges are deserted, doors are shut, and people are unfindable — in fact I still don’t know the names of the students who lived next door to me. This a bigger deal than people give it credit for: students are busy and campus is large, so you don’t really see someone unless you share a class, live near them, or date them. For example, I rarely talked to James Tao, even though we’d known each other for three years beforehand and had plenty in common.

Put more harshly: “Harvard’s dominant typical social tone is superficial, inane, and too frequently alcohol-drenched to be interesting. It actively thwarts any attempts to escape this atmosphere, by assigning groups of students to dorms randomly — thus guaranteeing all students a more-or-less uniformly superficial, inane and alcohol-drenched experience.”

The problems I mentioned were worse for me specifically since I took exclusively upper-level math courses. My classmates were all upperclassmen who all already knew each other and ate/lived elsewhere. For my own meals, the typical Annenberg conversation was either classes or gossip, so I had little to say to the other freshman (if I talked about my classes I sounded like a showoff). I was often sitting alone in my room, which was great for learning category theory but not so much for my mood. I ended up moving in to an MIT dorm for a good chunk of the school year, where it was much easier to find people I could relate well to (because they all lived in one place).

At Harvard I was constantly isolated and bored. I got sick of it and left.