Formal vs Functional Series (OR: Generating Function Voodoo Magic)

Epistemic status: highly dubious. I found almost no literature doing anything quite like what follows, which unsettles me because it makes it likely that I’m overcomplicating things significantly.

1. Synopsis

Recently I was working on an elegant problem which was the original problem 6 for the 2015 International Math Olympiad, which reads as follows:


[IMO Shortlist 2015 Problem C6] Let {S} be a nonempty set of positive integers. We say that a positive integer {n} is clean if it has a unique representation as a sum of an odd number of distinct elements from {S}. Prove that there exist infinitely many positive integers that are not clean.

Proceeding by contradiction, one can prove (try it!) that in fact all sufficiently large integers have exactly one representation as a sum of an even subset of {S}. Then, the problem reduces to the following:


Show that if {s_1 < s_2 < \dots} is an increasing sequence of positive integers and {P(x)} is a nonzero polynomial then we cannot have

\displaystyle \prod_{j=1}^\infty (1 - x^{s_j}) = P(x)

as formal series.

To see this, note that all sufficiently large {x^N} have coefficient {1 + (-1) = 0}. Now, the intuitive idea is obvious: the root {1} appears with finite multiplicity in {P} so we can put {P(x) = (1-x)^k Q(x)} where {Q(1) \neq 0}, and then we get that {1-x} on the RHS divides {P} too many times, right?

Well, there are some obvious issues with this “proof”: for example, consider the equality

\displaystyle 1 = (1-x)(1+x)(1+x^2)(1+x^4)(1+x^8) \dots.

The right-hand side is “divisible” by {1-x}, but the left-hand side is not (as a polynomial).

But we still want to use the idea of plugging {x \rightarrow 1^-}, so what is the right thing to do? It turns out that this is a complete minefield, and there are a lot of very subtle distinctions that seem to not be explicitly mentioned in many places. I think I have a complete answer now, but it’s long enough to warrant this entire blog post.

Here’s the short version: there’s actually two distinct notions of “generating function”, namely a “formal series” and “functional series”. They use exactly the same notation but are two different types of objects, and this ends up being the source of lots of errors, because “formal series” do not allow substituting {x}, while “functional series” do.

Spoiler: we’ll need the asymptotic for the partition function {p(n)}.

2. Formal Series {\neq} Functional Series

I’m assuming you’ve all heard the definition of {\sum_k c_kx^k}. It turns out unfortunately that this isn’t everything: there are actually two types of objects at play here. They are usually called formal power series and power series, but for this post I will use the more descriptive names formal series and functional series. I’ll do everything over {\mathbb C}, but one can of course use {\mathbb R} instead.

The formal series is easier to describe:

Definition 1

A formal series {F} is an infinite sequence {(a_n)_n = (a_0, a_1, a_2, \dots)} of complex numbers. We often denote it by {\sum a_nx^n = a_0 + a_1x + a_2x^2 + \dots}. The set of formal series is denoted {\mathbb C[ [x] ]}.

This is the “algebraic” viewpoint: it’s a sequence of coefficients. Note that there is no worry about convergence issues or “plugging in {x}”.

On the other hand, a functional series is more involved, because it has to support substitution of values of {x} and worry about convergence issues. So here are the necessary pieces of data:

Definition 2

A functional series {G} (centered at zero) is a function {G : U \rightarrow \mathbb C}, where {U} is an open disk centered at {0} or {U = \mathbb C}. We require that there exists an infinite sequence {(c_0, c_1, c_2, \dots)} of complex numbers satisfying

\displaystyle \forall z \in U: \qquad G(z) = \lim_{N \rightarrow \infty} \left( \sum_{k=0}^N c_k z^k \right).

(The limit is take in the usual metric of {\mathbb C}.) In that case, the {c_i} are unique and called the coefficients of {G}.

This is often written as {G(x) = \sum_n c_n x^n}, with the open set {U} suppressed.

Remark 3

Some remarks on the definition of functional series:

  • This is enough to imply that {G} is holomorphic (and thus analytic) on {U}.
  • For experts: note that I’m including the domain {U} as part of the data required to specify {G}, which makes the presentation cleaner. Most sources do something with “radius of convergence”; I will blissfully ignore this, leaving this data implicitly captured by {U}.
  • For experts: Perhaps non-standard, {U \neq \{0\}}. Otherwise I can’t take derivatives, etc.

Thus formal and functional series, despite having the same notation, have different types: a formal series {F} is a sequence, while a functional series {G} is a function that happens to be expressible as an infinite sum within its domain.

Of course, from every functional series {G} we can extract its coefficients and make them into a formal series {F}. So, for lack of better notation:

Definition 4

If {F = (a_n)_n} is a formal series, and {G : U \rightarrow \mathbb C} is a functional series whose coefficients equal {F}, then we write {F \simeq G}.

3. Finite operations

Now that we have formal and functional series, we can define sums. Since these are different types of objects, we will have to run definitions in parallel and then ideally check that they respect {\simeq}.

For formal series:

Definition 5

Let {F_1 = (a_n)_n} and {F_2 = (b_n)_n} be formal series. Then we set

\displaystyle \begin{aligned} (a_n)_n \pm (b_n)_n &= (a_n \pm b_n)_n \\ (a_n)_n \cdot (b_n)_n &= \left( \textstyle\sum_{j=0}^n a_jb_{n-j} \right)_n. \end{aligned}

This makes {\mathbb C[ [x] ]} into a ring, with identity {(0,0,0,\dots)} and {(1,0,0,\dots)}.

We also define the derivative {F = (a_n)_n} by {F' = ((n+1)a_{n+1})_n}.

It’s probably more intuitive to write these definitions as

\displaystyle \begin{aligned} \sum_n a_n x^n \pm \sum_n b_n x^n &= \sum_n (a_n \pm b_n) x^n \\ \left( \sum_n a_n x^n \right) \left( \sum_n b_n x^n \right) &= \sum_n \left( \sum_{j=0}^n a_jb_{n-j} \right) x^n \\ \left( \sum_n a_n x^n \right)' &= \sum_n na_n x^{n-1} \end{aligned}

and in what follows I’ll start to use {\sum_n a_nx^n} more. But officially, all definitions for formal series are in terms of the coefficients alone; these presence of {x} serves as motivation only.

Exercise 6

Show that if {F = \sum_n a_nx^n} is a formal series, then it has a multiplicative inverse if and only if {a_0 \neq 0}.

On the other hand, with functional series, the above operations are even simpler:

Definition 7

Let {G_1 : U \rightarrow \mathbb C} and {G_2 : U \rightarrow \mathbb C} be functional series with the same domain {U}. Then {G_1 \pm G_2} and {G_1 \cdot G_2} are defined pointwise.

If {G : U \rightarrow \mathbb C} is a functional series (hence holomorphic), then {G'} is defined poinwise.

If {G} is nonvanishing on {U}, then {1/G : U \rightarrow \mathbb C} is defined pointwise (and otherwise is not defined).

Now, for these finite operations, everything works as you expect:

Theorem 8 (Compatibility of finite operations)

Suppose {F}, {F_1}, {F_2} are formal series, and {G}, {G_1}, {G_2} are functional series {U \rightarrow \mathbb C}. Assume {F \simeq G}, {F_1 \simeq G_1}, {F_2 \simeq G_2}.

  • {F_1 \pm F_2 \simeq G_1 \pm G_2}, {F_1 \cdot F_2 = G_1 \cdot G_2}.
  • {F' \simeq G'}.
  • If {1/G} is defined, then {1/F} is defined and {1/F \simeq 1/G}.

So far so good: as long as we’re doing finite operations. But once we step beyond that, things begin to go haywire.

4. Limits

We need to start considering limits of {(F_k)_k} and {(G_k)_k}, since we are trying to make progress towards infinite sums and products. Once we do this, things start to burn.

Definition 9

Let {F_1 = \sum_n a_n x^n} and {F_2 = \sum_n b_n x^n} be formal series, and define the difference by

\displaystyle d(F_1, F_2) = \begin{cases} 2^{-n} & a_n \neq b_n, \; n \text{ minimal} \\ 0 & F_1 = F_2. \end{cases}

This function makes {\mathbb C[[x]]} into a metric space, so we can discuss limits in this space. Actually, it is a normed vector space obtained by {\left\lVert F \right\rVert = d(F,0)} above.

Thus, {\lim_{k \rightarrow \infty} F_k = F} if each coefficient of {x^n} eventually stabilizes as {k \rightarrow \infty}. For example, as formal series we have that {(1,-1,0,0,\dots)}, {(1,0,-1,0,\dots)}, {(1,0,0,-1,\dots)} converges to {1 = (1,0,0,0\dots)}, which we write as

\displaystyle \lim_{k \rightarrow \infty} (1 - x^k) = 1 \qquad \text{as formal series}.

As for functional series, since they are functions on the same open set {U}, we can use pointwise convergence or the stronger uniform convergence; we’ll say explicitly which one we’re doing.

Example 10 (Limits don’t work at all)

In what follows, {F_k \simeq G_k} for every {k}.

  • Here is an example showing that if {\lim_k F_k = F}, the functions {G_k} may not converge even pointwise. Indeed, just take {F_k = 1 - x^k} as before, and let {U = \{ z : |z| < 2 \}}.
  • Here is an example showing that even if {G_k \rightarrow G} uniformly, {\lim_k F_k} may not exist. Take {G_k = 1 - 1/k} as constant functions. Then {G_k \rightarrow 1}, but {\lim_k F_k} doesn’t exist because the constant term never stabilizes (in the combinatorial sense).
  • The following example from this math.SE answer by Robert Israel shows that it’s possible that {F = \lim_k F_k} exists, and {G_k \rightarrow G} pointwise, and still {F \not\simeq G}. Let {U} be the open unit disk, and set

    \displaystyle \begin{aligned} A_k &= \{z = r e^{i\theta} \mid 2/k \le r \le 1, \; 0 \le \theta \le 2\pi - 1/k\} \\ B_k &= \left\{ |z| \le 1/k \right\} \end{aligned}

    for {k \ge 1}. By Runge theorem there’s a polynomial {p_k(z)} such that

    \displaystyle |p_k(z) - 1/z^{k}| < 1/k \text{ on } A_k \qquad \text{and} \qquad |p_k(z)| < 1/k \text{ on }B_k.


    \displaystyle G_k(z) = z^{k+1} p_k(z)

    is the desired counterexample (with {F_k} being the sequence of coefficients from {G}). Indeed by construction {\lim_k F_k = 0}, since {\left\lVert F_k \right\rVert \le 2^{-k}} for each {k}. Alas, {|g_k(z) - z| \le 2/k} for {z \in A_k \cup B_k}, so {G_k \rightarrow z} converges pointwise to the identity function.

To be fair, we do have the following saving grace:

Theorem 11 (Uniform convergence and both limits exist is sufficient)

Suppose that {G_k \rightarrow G} converges uniformly. Then if {F_k \simeq G_k} for every {k}, and {\lim_k F_k = F}, then {F \simeq G}.

Proof: Here is a proof, copied from this math.SE answer by Joey Zhou. WLOG {G = 0}, and let {g_n(z) = \sum{a^{(n)}_kz^k}}. It suffices to show that {a_k = 0} for all {k}. Choose any {0<r<1}. By Cauchy’s integral formula, we have

\displaystyle \begin{aligned} \left|a_k - a^{(n)}_k\right| &= \left|\frac{1}{2\pi i} \int\limits_{|z|=r}{\frac{g(z)-g_n(z)}{z^{n+1}}\text{ d}z}\right| \\ & \le\frac{1}{2\pi}(2\pi r)\frac{1}{r^{n+1}}\max\limits_{|z|=r}{|g(z)-g_n(z)|} \xrightarrow{n\rightarrow\infty} 0 \end{aligned}

since {g_n} converges uniformly to {g} on {U}. Hence, {a_k = \lim\limits_{n\rightarrow\infty}{a^{(n)}_k}}. Since {a^{(n)}_k = 0} for {n\ge k}, the result follows. \Box

The take-away from this section is that limits are relatively poorly behaved.

5. Infinite sums and products

Naturally, infinite sums and products are defined by taking the limit of partial sums and limits. The following example (from math.SE again) shows the nuances of this behavior.

Example 12 (On {e^{1+x}})

The expression

\displaystyle \sum_{n=0}^\infty \frac{(1+x)^n}{n!} = \lim_{N \rightarrow \infty} \sum_{n=0}^N \frac{(1+x)^n}{n!}

does not make sense as a formal series: we observe that for every {N} the constant term of the partial sum changes.

But this does converge (uniformly, even) to a functional series on {U = \mathbb C}, namely to {e^{1+x}}.

Exercise 13

Let {(F_k)_{k \ge 1}} be formal series.

  • Show that an infinite sum {\sum_{k=1}^\infty F_k(x)} converges as formal series exactly when {\lim_k \left\lVert F_k \right\rVert = 0}.
  • Assume for convenience {F_k(0) = 1} for each {k}. Show that an infinite product {\prod_{k=0}^{\infty} (1+F_k)} converges as formal series exactly when {\lim_k \left\lVert F_k-1 \right\rVert = 0}.

Now the upshot is that one example of a convergent formal sum is the expression {\lim_{N} \sum_{n=0}^N a_nx^n} itself! This means we can use standard “radius of convergence” arguments to transfer a formal series into functional one.

Theorem 14 (Constructing {G} from {F})

Let {F = \sum a_nx^n} be a formal series and let

\displaystyle r = \frac{1}{\limsup_n \sqrt[n]{|c_n|}}.

If {r > 0} then there exists a functional series {G} on {U = \{ |z| < r \}} such that {F \simeq G}.

Proof: Let {F_k} and {G_k} be the corresponding partial sums of {c_0x^0} to {c_kx^k}. Then by Cauchy-Hadamard theorem, we have {G_k \rightarrow G} uniformly on (compact subsets of) {U}. Also, {\lim_k F_k = F} by construction. \Box

This works less well with products: for example we have

\displaystyle 1 \equiv (1-x) \prod_{j \ge 0} (1+x^{2^j})

as formal series, but we can’t “plug in {x=1}”, for example,

6. Finishing the original problem

We finally return to the original problem: we wish to show that the equality

\displaystyle P(x) = \prod_{j=1}^\infty (1 - x^{s_j})

cannot hold as formal series. We know that tacitly, this just means

\displaystyle \lim_{N \rightarrow \infty} \prod_{j=1}^N\left( 1 - x^{s_j} \right) = P(x)

as formal series.

Here is a solution obtained only by only considering coefficients, presented by Qiaochu Yuan from this MathOverflow question.

Both sides have constant coefficient {1}, so we may invert them; thus it suffices to show we cannot have

\displaystyle \frac{1}{P(x)} = \frac{1}{\prod_{j=1}^{\infty} (1 - x^{s_j})}

as formal power series.

The coefficients on the LHS have asymptotic growth a polynomial times an exponential.

On the other hand, the coefficients of the RHS can be shown to have growth both strictly larger than any polynomial (by truncating the product) and strictly smaller than any exponential (by comparing to the growth rate in the case where {s_j = j}, which gives the partition function {p(n)} mentioned before). So the two rates of growth can’t match.

New algebra handouts on my website

For olympiad students: I have now published some new algebra handouts. They are:

  • Introduction to Functional Equations, which cover the basic techniques and theory for FE’s typically appearing on olympiads like USA(J)MO.
  • Monsters, an advanced handout which covers functional equations that have pathological solutions. It covers in detail the solutions to Cauchy functional equation.
  • Summation, which is a compilation of various types of olympiad-style sums like generating functions and multiplicative number theory.

I have also uploaded:

  • English, notes on proof-writing that I used at the 2016 MOP (Mathematical Olympiad Summer Program).

You can download all these (and other handouts) from my MIT website. Enjoy!

Miller-Rabin (for MIT 18.434)

This is a transcript of a talk I gave as part of MIT’s 18.434 class, the “Seminar in Theoretical Computer Science” as part of MIT’s communication requirement. (Insert snarky comment about MIT’s CI-* requirements here.) It probably would have made a nice math circle talk for high schoolers but I felt somewhat awkward having to present it to a bunch of students who were clearly older than me.

1. Preliminaries

1.1. Modular arithmetic

In middle school you might have encountered questions such as

Exercise 1

What is {3^{2016} \pmod{10}}?

You could answer such questions by listing out {3^n} for small {n} and then finding a pattern, in this case of period {4}. However, for large moduli this “brute-force” approach can be time-consuming.

Fortunately, it turns out that one can predict the period in advance.

Theorem 2 (Euler’s little theorem)

  1. Let {\gcd(a,n) = 1}. Then {a^{\phi(n)} \equiv 1 \pmod n}.
  2. (Fermat) If {p} is a prime, then {a^p \equiv a \pmod p} for every {a}.

Proof: Part (a) is a special case of Lagrange’s Theorem: if {G} is a finite group and {g \in G}, then {g^{|G|}} is the identity element. Now select {G = (\mathbb Z/n\mathbb Z)^\times}. Part (b) is the case {n=p}. \Box

Thus, in the middle school problem we know in advance that {3^4 \equiv 1 \pmod{10}} because {\phi(10) = 4}. This bound is sharp for primes:

Theorem 3 (Primitive roots)

For every {p} prime there’s a {g \pmod p} such that {g^{p-1} \equiv 1 \pmod p} but {g^{k} \not\equiv 1 \pmod p} for any {k < p-1}. (Hence {(\mathbb Z/p\mathbb Z)^\times \cong \mathbb Z/(p-1)}.)

For a proof, see the last exercise of my orders handout.

We will define the following anyways:

Definition 4

We say an integer {n} (thought of as an exponent) annihilates the prime {p} if

  • {a^n \equiv 1 \pmod p} for every {a \not\equiv 0 \pmod p},
  • or equivalently, {p-1 \mid n}.

Theorem 5 (All/nothing)

Suppose an exponent {n} does not annihilate the prime {p}. Then more than {\frac{1}{2} p} of {x \pmod p} satisfy {x^n \not\equiv 1 \pmod p}.

Proof: Much stronger result is true: in {x^n \equiv 1 \pmod p} then {x^{\gcd(n,p-1)} \equiv 1 \pmod p}. \Box

1.2. Repeated Exponentiation

Even without the previous facts, one can still do:

Theorem 6 (Repeated exponentation)

Given {x} and {n}, one can compute {x^n \pmod N} with {O(\log n)} multiplications mod {N}.

The idea is that to compute {x^{600} \pmod N}, one just multiplies {x^{512+64+16+8}}. All the {x^{2^k}} can be computed in {k} steps, and {k \le \log_2 n}.

1.3. Chinese remainder theorem

In the middle school problem, we might have noticed that to compute {3^{2016} \pmod{10}}, it suffices to compute it modulo {5}, because we already know it is odd. More generally, to understand {\pmod n} it suffices to understand {n} modulo each of its prime powers.

The formal statement, which we include for completeness, is:

Theorem 7 (Chinese remainder theorem)

Let {p_1}, {p_2}, \dots, {p_m} be distinct primes, and {e_i \ge 1} integers. Then there is a ring isomorphism given by the natural projection

\displaystyle \mathbb Z/n \rightarrow \prod_{i=1}^m \mathbb Z/p_i^{e_i}.

In particular, a random choice of {x \pmod n} amounts to a random choice of {x} mod each prime power.

For an example, in the following table we see the natural bijection between {x \pmod{15}} and {(x \pmod 3, x \pmod 5)}.

\displaystyle \begin{array}{c|cc} x \pmod{15} & x \pmod{3} & x \pmod{5} \\ \hline 0 & 0 & 0 \\ 1 & 1 & 1 \\ 2 & 2 & 2 \\ 3 & 0 & 3 \\ 4 & 1 & 4 \\ 5 & 2 & 0 \\ 6 & 0 & 1 \\ 7 & 1 & 2 \end{array} \quad \begin{array}{c|cc} x \pmod{15} & x \pmod{3} & x \pmod{5} \\ \hline 8 & 2 & 3 \\ 9 & 0 & 4 \\ 10 & 1 & 0 \\ 11 & 2 & 1 \\ 12 & 0 & 2 \\ 13 & 1 & 3 \\ 14 & 2 & 4 \\ && \end{array}

2. The RSA algorithm

This simple number theory is enough to develop the so-called RSA algorithm. Suppose Alice wants to send Bob a message {M} over an insecure channel. They can do so as follows.

  • Bob selects integers {d}, {e} and {N} (with {N} huge) such that {N} is a semiprime and

    \displaystyle de \equiv 1 \pmod{\phi(N)}.

  • Bob publishes both the number {N} and {e} (the public key) but keeps the number {d} secret (the private key).
  • Alice sends the number {X = M^e \pmod N} across the channel.
  • Bob computes

    \displaystyle X^d \equiv M^{de} \equiv M^1 \equiv M \pmod N

    and hence obtains the message {M}.

In practice, the {N} in RSA is at least {2000} bits long.

The trick is that an adversary cannot compute {d} from {e} and {N} without knowing the prime factorization of {N}. So the security relies heavily on the difficulty of factoring.

Remark 8

It turns out that we basically don’t know how to factor large numbers {N}: the best known classical algorithms can factor an {n}-bit number in

\displaystyle O\left( \exp\left( \frac{64}{9} n \log(n)^2 \right)^{1/3} \right)

time (“general number field sieve”). On the other hand, with a quantum computer one can do this in {O\left( n^2 \log n \log \log n \right)} time.

3. Primality Testing

Main question: if we can’t factor a number {n} quickly, can we at least check it’s prime?

In what follows, we assume for simplicity that {n} is squarefree, i.e. {n = p_1 p_2 \dots p_k} for distinct primes {p_k}, This doesn’t substantially change anything, but it makes my life much easier.

3.1. Co-RP

Here is the goal: we need to show there is a random algorithm {A} which does the following.

  • If {n} is composite then
    • More than half the time {A} says “definitely composite”.
    • Occasionally, {A} says “possibly prime”.
  • If {n} is prime, {A} always says “possibly prime”.

If there is a polynomial time algorithm {A} that does this, we say that PRIMES is in Co-RP. Clearly, this is a very good thing to be true!

3.2. Fermat

One idea is to try to use the converse of Fermat’s little theorem: given an integer {n}, pick a random number {x \pmod n} and see if {x^{n-1} \equiv 1 \pmod n}. (We compute using repeated exponentiation.) If not, then we know for sure {n} is not prime, and we call {x} a Fermat witness modulo {n}.

How good is this test? For most composite {n}, pretty good:

Proposition 9

Let {n} be composite. Assume that there is a prime {p \mid n} such that {n-1} does not annihilate {p}. Then over half the numbers mod {n} are Fermat witnesses.

Proof: Apply the Chinese theorem then the “all-or-nothing” theorem. \Box
Unfortunately, if {n} doesn’t satisfy the hypothesis, then all the {\gcd(x,n) = 1} satisfy {x^{n-1} \equiv 1 \pmod n}!

Are there such {n} which aren’t prime? Such numbers are called Carmichael numbers, but unfortunately they exist, the first one is {561 = 3 \cdot 11 \cdot 17}.

Remark 10

For {X \gg 1}, there are more than {X^{1/3}} Carmichael numbers at most {X}.

Thus these numbers are very rare, but they foil the Fermat test.

Exercise 11

Show that a Carmichael number is not a semiprime.

3.3. Rabin-Miller

Fortunately, we can adapt the Fermat test to cover Carmichael numbers too. It comes from the observation that if {n} is prime, then {a^2 \equiv 1 \pmod n \implies a \equiv \pm 1 \pmod n}.

So let {n-1 = 2^s t}, where {t} is odd. For example, if {n = 561} then {560 = 2^4 \cdot 35}. Then we compute {x^t}, {x^{2t}}, \dots, {x^{n-1}}. For example in the case {n=561} and {x=245}:

\displaystyle \begin{array}{c|r|rrr} & \mod 561 & \mod 3 & \mod 11 & \mod 17 \\ \hline x & 245 & -1 & 3 & 7 \\ \hline x^{35} & 122 & -1 & \mathbf 1 & 3 \\ x^{70} & 298 & \mathbf 1 & 1 & 9 \\ x^{140} & 166 & 1 & 1 & -4 \\ x^{280} & 67 & 1 & 1 & -1 \\ x^{560} & 1 & 1 & 1 & \mathbf 1 \end{array}

And there we have our example! We have {67^2 \equiv 1 \pmod{561}}, so {561} isn’t prime.

So the Rabin-Miller test works as follows:

  • Given {n}, select a random {x} and compute powers of {x} as in the table.
  • If {x^{n-1} \not\equiv 1}, stop, {n} is composite (Fermat test).
  • If {x^{n-1} \equiv 1}, see if the entry just before the first {1} is {-1}. If it isn’t then we say {x} is a RM-witness and {n} is composite.
  • Otherwise, {n} is “possibly prime”.

How likely is probably?

Theorem 12

If {n} is Carmichael, then over half the {x \pmod n} are RM witnesses.

Proof: We sample {x \pmod n} randomly again by looking modulo each prime (Chinese theorem). By the theorem on primitive roots, show that the probability the first {-1} appears in any given row is {\le \frac{1}{2}}. This implies the conclusion. \Box

Exercise 13

Improve the {\frac{1}{2}} in the problem to {\frac34} by using the fact that Carmichael numbers aren’t semiprime.

3.4. AKS

In August 6, 2002, it was in fact shown that PRIMES is in P, using the deterministic AKS algorithm. However, in practice everyone still uses Miller-Rabin since the implied constants for AKS runtime are large.

Mechanism Design and Revenue Equivalence

Happy Pi Day! I have an economics midterm on Wednesday, so here is my attempt at studying.

1. Mechanisms

The idea is as follows.

  • We have {N} people and a seller who wants to auction off a power drill.
  • The {i}th person has a private value of at most {\$1000} on the power drill. We denote it by {x_i \in [0,1000]}.
  • However, everyone knows the {x_i} are distributed according to some measure {\mu_i} supported on {[0, 1000]}. (let’s say a Radon measure, but I don’t especially care). Tacitly we assume {\mu_i([0,1000]) = 1}.

Definition 1

Consider a game {M} played as follows:

  • Each player {i=1, \dots, N} makes a bid {b_i} (which depends on how much they value the object)
  • Based on all the bids {\vec b = \left( b_1, \dots, b_N \right)}, each player has a chance {Q_i(\vec b) \in [0,1]} of actually obtaining the object. We call {Q = \{Q_i\}_{i=1}^N} the allocation function and require {\sum_i Q_i(\vec b) \le 1}.
  • Based on all the bids {\vec b = \left( b_1, \dots, b_N \right)}, each player makes a payment {P_i(\vec b) \in \mathbb R_{\ge 0}}. We call the {P = \{P_i\}_{i=1}^n} the payment function. Note that one might have to pay even if they don’t get the drill!
  • The utility of the {i}th player is

    \displaystyle  U_i(\vec b) = Q_i(\vec b) \cdot x_i - P_i(\vec b)

    i.e. the expected chance they get the power drill minus the amount which they pay.

We call the pair {(P,Q,\mu)} a mechanism.

For experts: we require that each {P_i} and {Q_i} is measurable. Right now this is not a very good definition, because there are no assumptions on what {P} and {Q} look like. Nonetheless, we’ll give some examples.

2. Examples of mechanisms

In the auction that you’d see in real life, we usually set

\displaystyle  Q = Q_{\text{highest}} \overset{\mathrm{def}}{=} \text{highest bidder wins}

which is the simple rule that the highest bidder gets the power drill with probability {1}; if there is a tie, we pick one of the highest bidders at random.

In all the examples that follow, for simplicity let’s take the case {N=2}, and call the two players Anna and Elsa. We assume that both Anna and Elsa the value they place on the drill is uniform between {[0,1000]}. Finally, we give the auctioneer a name, say Hans.

2.1. First-price auction

The first-price auction is the one you’ve probably heard of: each player makes a bid and

  • {Q = Q_{\text{highest}}}, and
  • {P} is defined by requiring the winner to pay their bid.

How do Anna and Elsa behave in this auction? Clearly no one will bid more than they think the drill is worth, because then they stand to lose utility if they happen to win the auction. But the truth is they actually will bid less than they think the drill is worth.

For concreteness, suppose Anna values the drill at {\$700}. It doesn’t make sense for Anna to bid more than {\$700} obviously. But perhaps she should bid {\$699} — save a dollar. After all, what’s the chance that Elsa would bid right between {\$700} and {\$699}? For that matter, Anna knows that Elsa has a {50\%} chance of valuing the drill at less than {\$500}. So if Anna bids {\$500}, she has at least a {50\%} of saving at least {\$200}; it makes no sense for her to bid her true {\$700} value.

It gets better. Anna knows that Elsa isn’t stupid, and isn’t going to bid {\$500} even if her true value is {\$500}. That is, Elsa is going to try and sell Hans short as well. Given this Anna can play the game of cheating Hans more aggressively, and so an ad infinitum.

Of course there’s a way to capture this idea of “I know that you know that I know that you know. . .”: we just compute the Nash equilibrium.

Proposition 2 (Nash eqilibirum of the first-price auction for {N=2})

Consider a first-price auction where Anna and Elsa have values uniformly distributed in {[0, 1000]}. Suppose both Anna and Elsa bid {x/2} if they have value {x}. Then this is a Nash equilibrium.

Proof: Suppose Anna values the drill at {a} and wants to make a bid {x}. Elsa values the drill at {e}, and follows the equilibrium by bidding {e/2}. For a bid of {x}, Anna gets utility

\displaystyle  \begin{cases} a-x & x>e/2 \\ 0 & \text{otherwise} \end{cases}.

The probability of winning with {x} is thus {\min(1, 2x/1000)} (this the probability that {e < 2x}) so the expected utility is

\displaystyle  (a-x)\min\left( 1, \frac{2x}{1000} \right) \le \frac{2}{1000} x(a-x).

Hence we see {x = a/2} maximizes Anna’s expected utility. \Box

The first-price auction is interesting because both players “lie” when bidding in the Nash equilibrium. For this reason we say that the first-price auction is not incentive compatible.

Just for interest, let’s compute how much money Hans is going to make off the drill in this equilibrium. The amount paid to him is equal to

\displaystyle  \mathbf E_{a,e} \left( \max(a/2, e/2) \right) = \frac{1000}{3}.

To see this we had to use the fact that if two numbers in {[0,1]} are chosen at random, the expected value of the larger is {\frac23}. Multiplying by {1000/2 = 500} gives the answer: Hans expects to make {\$333}.

2.2. Second-price auction

The second-price auction is the other one you’ve probably heard of: each player makes a bid and

  • {Q = Q_{\text{highest}}}, and
  • {P} is defined by requiring the winner to pay the smallest amount needed to win, i.e. the second highest bid.

The fundamental difference is that in a second-price auction, a player “doesn’t need to be shy”: if they place a large bid, they don’t have to worry about possibly paying it.

Another way to think about is as a first-price auction with the property that the winning player can retroactively change their bid, provided they still win the auction. So unlike before there is no advantage to being honest.

Indeed, the second-price auction is incentive compatible in a very strong sense: bidding your true value is the best thing to do regardless of whether your opponents are playing optimally.

Proposition 3 (Second-price auctions are incentive compatible)

In a second-price auction, bidding truthfully is a weakly dominant strategy.

Proof: Easy. Check it. \Box

Just for interest, let’s compute how much money Hans is going to make off the drill in this equilibrium. This time he amount paid to him is equal to

\displaystyle  \mathbf E_{a,e} \left( \min(a, e) \right) = \frac{1000}{3}.

Here we had to use the fact that if two numbers in {[0,1]} are chosen at random, the expected value of the smaller is {\frac13}. This might come as a surprise: the expected revenue is {\$333} in this auction too.

2.3. All-pay auction

The all-pay auction is like lobbying. Each player makes a bid, and

  • {Q = Q_{\text{highest}}}, and
  • {P} is defined by requiring everyone to pay their bid, regardless of whether they win the power drill or not.

This is clearly not incentive compatible. In fact, the Nash equilibrium is as follows:

Proposition 4 (Nash equilibirum of the all-pay auction)

Consider a first-price auction where Anna and Elsa have values uniformly distributed in {[0, 1000]}. Suppose both Anna and Elsa bid {\frac{1}{2} \cdot 1000(x/1000)^2} if they have value {x}. Then this is a Nash equilibrium.

Proof: Omitted, but a fun and not-hard exercise if you like integrals. It will follow from a later result. \Box

Just for interest, let’s compute how much money Hans is going to make off the drill in this equilibrium. This time he amount paid to him is equal to

\displaystyle  \mathbf E_{a,e} \left( \frac{a^2}{2000} + \frac{b^2}{2000} \right) = \int_{0}^{1000} \frac{x^2}{1000} \; dx = \frac{1000}{3}.

Surprise — same value again! This is a very special case of the Revenue Equivalence Theorem later.

2.4. Extortion

We’ve seen three examples that all magically gave {\$333} as the expected gain. So here’s a silly counterexample to show that not every auction is going to give {\$333} as an expected gain. It blatantly abuses the fact that we’ve placed almost no assumptions on {P} and {Q}:

  • {Q = Q_{\text{highest}}}, or any other {Q} for that matter, and
  • {P} is defined by requiring both Anna and Elsa to give {\$1000000} to Hans.

This isn’t very much an auction at all: more like Hans just extracting money from Anna and Elsa. Hans is very happy with this arrangement, Anna and Elsa not so much. So we want an assumption on our auctions to prevent this silly example:

Definition 5

A mechanism {(M, \sigma)} is voluntary (or individually rational) if {u_i(x_i) \ge 0} for every {x_i \in [0,1000]}.

2.5. Second-price auction with reserve

Here is a less stupid example of how Hans can make more money. The second-price auction with reserve is the same as the second-price auction, except Hans himself also places a bid of {R = 500}. Thus if no one bids more than {R}, the item is not sold.

For the same reason as in the usual second-price auction, bidding truthfully is optimal for each players. The cases are:

  • If both Anna and Elsa bid less than {\$500}, no one gains anything.
  • If both Anna and Elsa bids more than {\$500}, the higher bidder wins and pays the lower bid.
  • If exactly one player bids more than {\$500}, that player wins and pays the bid for {\$500}.

So Hans suffers some loss in the first case, but earns some extra money in the last case (when compared to the traditional second-price auction.) It turns out that if you do the computation, then Hans gets an expected profit of

\displaystyle  \frac{1250}{3} \approx \$417

meaning he earns another {\$80} or so by setting a reserve price.

3. Direct mechanisms

As it stands, {b_i} might depend in complicated ways on the actual values {x_i}: for example in the first-price auction. We can capture this formalism as follows.

Definition 6

A direct mechanism is a pair {M = (M, \sigma)} where

  • {(P,Q, \mu)} is a mechanism,
  • {\sigma = \{\sigma_i\}_{i=1}^N} is a Nash equilibrium of bidding strategies for the bidders. So in this equilibrium the {i}th player will bid {\sigma_i(x_i)}.

If {\sigma = \mathrm{id}}, meaning {\sigma_i(x) \equiv x} for every {i}, then we say {M} is incentive compatible.

So in other words, I’m equipping the mechanism {M} with a particular Nash equilibrium {\sigma}. This is not standard, but I think it is harder to state the theorems in a non-confusing form otherwise.

Definition 7

Let {M = (M, \sigma)} be a direct mechanism. Then we define {p_i(x)}, {q_i(x)}, {u_i(x)} by

\displaystyle  u_i(x) = \mathbb E\left[ U_i(\vec b) \mid x_i = x \text{ and everyone follows } \sigma \right]

For example, {u_1(x)} is the expected utility of the 1st player conditioned on them having value {x} for the power drill:

\displaystyle  u_1(x) = \int_{x_2=0}^{1000} \dots \int_{x_N=0}^{1000} U_1(\sigma_1(x), \sigma_2(x_2), \dots, \sigma_N(x_N)) \; d\mu_2 \dots d\mu_N.

Similarly, let

\displaystyle  \begin{aligned} p_i(x) &= \mathbb E\left[ P_i(\vec b) \mid x_i = x \text{ and everyone follows }\sigma \right] \\ q_i(x) &= \mathbb E\left[ Q_i(\vec b) \mid x_i = x \text{ and everyone follows }\sigma \right]. \end{aligned}

Note that {p_i}, {q_i}, {u_i} depend not only on the mechanism {M} itself but also on the attached equilibrium {\sigma}.

It’s important to realize the functions {p_i}, {q_i}, {u_i} carry much more information than just {P}, {Q}, {U}. All three functions depend on the equilibrium strategy {\sigma}, which in turn depend on both {P} and {Q}. Moreover, all three functions also depend on {\mu}. Hence for example {q} actually depends indirectly on {P} as well, because the choice of {P} affects the resulting equilibrium {\sigma}.

Example 8 (Example of {\sigma}, {q_i}, {p_i})

Let’s take the first-price auction with two players Anna and Elsa. If we call it {M = (M, \sigma)} then as we described before we have:

  • {N = 2},
  • {\mu_1} and {\mu_2} are uniform distributions over {[0,1000]},
  • {P_i} is defined by having player {i} pay their bid upon winning.
  • {Q = Q_{\text{highest}}}

Moreover, the Nash equilibrium {\sigma = (\sigma_1, \sigma_2)} is given by

\displaystyle  \sigma_i(x) = x/2

for all {x}, as we saw earlier. Consequently, we have

  • {q_i(x) = x/1000} since the probability of winning (and hence winning the bid) is proportional to the value placed on the item (since {\mu} is a uniform distribution).
  • {p_i(x) = x/2 \cdot x/1000} as the expected payment: there’s a {x/1000} chance of winning, and a payment of {x/2} if you do.

4. Equivalence of utility and payment

In what follows let {i} be any index.

We now prove that

Lemma 9 (Envelope theorem)

Assume {M = (M, \sigma)} is a direct mechanism. Then {u} is convex, and

\displaystyle  u_i'(x) = q_i(x)

except for at most countably many points at which {u} is not differentiable.

Proof: Since {\sigma} is an equilibrium, we have

\displaystyle  x \cdot q_i(x) - p_i(x) = u_i(x) \ge x \cdot q_i(b) - p_i(b) \qquad \forall b \in I.

i.e. there is no benefit in lying and bidding {b} rather than {x}.

First, let’s show that if {x} is differentiable, then {u_i'(x) = q_i(x)}. We have that

\displaystyle  \lim_{h \rightarrow 0} \frac{u_i(x+h)-u_i(x)}{h} \ge \lim_{h \rightarrow 0} \frac{\left[ (x+h)q_i(x)-p_i(x) \right] - \left[ x \cdot q_i(x)-p_i(x) \right]}{h} = q(x).


\displaystyle  \lim_{h \rightarrow 0} \frac{u_i(x)-u_i(x-h)}{h} \le \lim_{h \rightarrow 0} \frac{\left[ x \cdot q_i(x)-p_i(x) \right] - \left[ (x-h)q_i(x)-p_i(x) \right]}{h} = q_i(x).

This implies the limit, if it exists, must be {q_i(x)}. We’ll omit the proof that {u} is differentiable almost everywhere, remarking that it follows from {q_i(x)} being nondecreasing in {x} (proved later). \Box

Theorem 10 (Utility equivalence)

Let {(M, \sigma)} be a direct mechanism. For any {x},

\displaystyle  u_i(x) = u_i(0) + \int_0^x q_i(t) \; dt.

Proof: Fundamental theorem of calculus. \Box

Theorem 11 (Payment equivalence)

Let {(M, \sigma)} be a direct mechanism. For any {x},

\displaystyle  p_i(x) = p_i(0) + x \cdot q_i(x) - \int_0^x q_i(t) \; dt.

Thus {p} is determined by {q} up to a constant shift.

Proof: Use {u_i(x) = x \cdot q_i(x) - p_i(x)} and {u_i(0) = - p_i(0)}. \Box

This means that both functions {p} and {u} are completely determined, up to a constant shift, by the expected allocation {q} in the equilibrium {\sigma}.

5. Revenue equivalence

A corollary:

Corollary 12 (Revenue equivalence)

Let {M = (M, \sigma)} be a mechanism. Then the expected revenue of the auctioneer, namely,

\displaystyle  \mathbf E_{\vec x} \sum_{i=1}^N p_i(x_i)

depends only on {q_i} and {p_i(0)}.

Very often, textbooks will add the additional requirement that {u_i(0) = 0} or {p_i(0) = 0}, in which case the statements become slightly simpler, as the constant shifts go away.

Here are the two important corollaries, which as far as I can tell are never both stated.

Corollary 13 (Revenue equivalence for incentive compatible mechanisms)

If {M = (M, \mathrm{id})} is incentive compatible, then {u_i(x)} and {p_i(x)} (and hence the seller’s revenue) depends only on the allocation function {Q} and distributions {\mu}, up to a constant shift.

Proof: In an incentive compatible situation where {\sigma_i(x) = x} we have

\displaystyle  q_1(x) = \int_{x_2=0}^{1000} \dots \int_{x_N=0}^{1000} Q_1(x, x_2, \dots, x_N) \; d\mu_2 d\mu_3 \dots d\mu_N

so {q_1} depends only on {Q} and {\mu} up to a constant shift. Ditto for any other {q_i}. \Box

Corollary 14 (Revenue equivalence for {Q_{\text{highest}}} auctions)

Suppose {M = (M, \sigma)} is a mechanism in which

  • The allocation function is {Q = Q_{\text{highest}}},
  • The {\sigma_i(x)} is strictly increasing in {x} for all {i} (players with higher values bid more).

Then {u(x)} and {p(x)} are determined up to constant shifts by {\mu}.

Proof: By the assumption on {\sigma} we have

\displaystyle  Q_{\text{highest}}( \sigma_1(x_1), \dots, \sigma_N(x_N) ) = Q_{\text{highest}}\left( x_1, \dots, x_N \right)

so it follows for example that

\displaystyle  \begin{aligned} q_1(x) &= \int_{x_2=0}^{1000} \dots \int_{x_N=0}^{1000} Q_1(\sigma_1(x), \sigma_2(x_2), \dots, \sigma_N(x_N)) \; d\mu_2 \dots d\mu_N \\ &= \int_{x_2=0}^{1000} \dots \int_{x_N=0}^{1000} Q_1(x, x_2, \dots, x_N) \; d\mu_2 \dots d\mu_N. \end{aligned}

Once again {q_i} now depends only on {\mu}. \Box

As an application, we can actually use the revenue equivalence theorem to compute the equilibrium strategies of the first-price, second-price, and all-pay auctions with {n \ge 2} players.

Corollary 15 (Nash equilibria of common {Q_{\text{highest}}} auctions)

Suppose a player has value {x \in [0,1000]} as in our setup, and that the prior {\mu} is distributed uniformly. Each of the following is a Nash equilibrium:

  • In a first-price auction, bid {\frac{n}{n+1} x}.
  • In a second-price auction, bid {x} (i.e. bid truthfully).
  • In an all-pay auction, bid {1000 \cdot \frac{n-1}{n} (x/1000)^n}.

Proof: First, as we saw already the second-price auction has an equilibrium where everyone bids truthfully. In this case, the probability of winning is {(x/1000)^{n-1}} and the expected payment when winning is {\frac{n-1}{n} x} (this is the expected value of the largest of {n-1} numbers in {[0,x]}.) Now by revenue equivalence, we have

\displaystyle  p_i^{\mathrm{all}}(x) = p_i^{\mathrm{I}}(x) = p_i^{\mathrm{II}}(x) = 1000 \cdot \frac{n-1}{n} \left( x/1000 \right)^n.

Now we examine the all-pay and first-price auction.

  • We have {p_i^{\mathrm{all}}(x) = 1000 \cdot \frac{n-1}{n} \left( x/1000 \right)^n}, i.e. in the equilibrium strategy for the all-pay auction, a player with type {x} pays on average {1000 \cdot \frac{n-1}{n} \left( x/1000 \right)^n}. But the payment in an all-pay auction is always the bid! Hence conclusion.
  • We have {p_i^{\mathrm{I}}(x) = p_i^{\mathrm{II}}(x)}, and since in both cases the chance of paying at all is {(x/1000)^{n-1}}, the payment if a player does win is {\frac{n-1}{n} x}; hence the equilibrium strategy is to bid {\frac{n-1}{n}x}.


6. Design feasibility

By now we have seen that all our mechanisms really depend mostly on the functions {q_i}, (from which we can then compute {p_i} and {u_i}) while we have almost completely ignored the parameters {P_i}, {Q_i}, {\sigma} which give rise to the mechanism {(M, \sigma)} in the first place.

We would like to continue doing this, and therefore, we want a way to reverse the work earlier: given the functions {q_i} construct a mechanism {(M, \sigma)} with those particular expected allocations. The construction need not even be explicit; we will be content just knowing such a mechanisms exists.

Thus, the critical question is: which functions {q_i} can actually arise? The answer is that the only real constraint is that {q_i} are nondecreasing.

Theorem 16 (Feasability rule)

Consider {N} players with a fixed a distribution {\{\mu_i\}_{i=1}^N} of private values. Consider any measurable functions {q_1, \dots, q_N : [0,1000] \rightarrow \mathbb R_{\ge 0}}.

Then there exists a direct mechanism {(M, \sigma)} with {q_i} as the expected allocations if and only if

  • each function {q_i} is nondecreasing.
  • {q_1(x_1) + \dots + q_N(x_N) \le 1} with probability {1}.

Proof: First, we show that any {q_i} arising from a direct mechanism are nondecreasing. Indeed, if {x > y} then we have the inequalities

\displaystyle  \begin{aligned} x \cdot q_i(x) - p_i(x) &\ge x \cdot q_i(y) - p_i(y) \\ y \cdot q_i(y) - p_i(y) &\ge y \cdot q_i(x) - p_i(x). \end{aligned}

If we add these inequalities we recover {(x-y)q_i(x) \ge (x-y)q_i(y)}, which shows that {q_i} is nondecreasing. The second condition just reflects that {\sum Q_i(\vec b) \le 1}.

Conversely, suppose {q_i} are given nondecreasing function. We will construct an incentive compatible mechanism inducing them. First, define {p_i(x) = x \cdot q_i(x) - \int_0^x q_i(t) \; dt} as predicted by Revenue Equivalence. First, define {M = (P,Q, \mu)} by

  • {P_i(x_1, \dots, x_N) = p_i(x_i)}, and
  • {Q_i(x_1, \dots, x_N) = q_i(x_i)}. (Note {\sum_i Q_i(x_i) \le 1}.)

Then trivially {p_i^M = p_i} and {q_i^M = q_i}.

However, we haven’t specified a {\sigma} yet! This is the hard part, but the crucial claim is that we can pick {\sigma = \mathrm{id}}: that is, {(M, \mathrm{id})} is an incentive compatible direct mechanism.

Thus we need to check that for all {x} and {y} that

\displaystyle  u_i(x) \ge x \cdot q_i(y) - p_i(y)

We just do {x > y} since the other case is analogous; then the inequality we want to prove rearranges as

\displaystyle  \begin{aligned} \iff u_i(x) - u_i(y) &\ge (x-y) \cdot q_i(y) \\ \iff \int_y^x q(t) \; dt &\ge \int_y^x q(y) \; dt \end{aligned}

Since {q} is increasing, this is immediate. \Box

In particular, we can now state:

Corollary 17 (Individually Rational)

A direct mechanism {(M, \sigma)} is voluntary if and only if

\displaystyle  u_i(0) \ge 0 \iff p_i(0) \le 0

for every {i}.

Proof: Since {u_i' = q_i \ge 0}, the function {u_i} is nonnegative everywhere if and only if {u_i(0) \ge 0}. \Box

7. Optimal auction

Since we’re talking about optimal auctions, we now restrict our attention to auctions in which {p_i(0) = 0} for every {i} (hence the auction is voluntary).

Now, we want to find the amount of money we expect to extract for player {i}. Let’s compute the expected revenue given a function {q} and distribution {\mu}. Of course, we know that

\displaystyle  \mathbf E_{x_i}\left[ p_i(x_i) \right] = \int_{t=0}^{1000} p_i(t) \; d\mu_i

but we also know by revenue equivalence that we want to instead use the function {q_i}, rather than the function {p_i}. So let’s instead use our theorem for the integral to compute it:

\displaystyle  \mathbf E_{x_i}\left[ p_i(x_i) \right] = p_i(0) + \mathbf E_{x_i}\left[ x_i \cdot q_i(x_i) \right] - \mathbf E_{x_i}\left[ \int_0^{x_i} q(t) \; dt \right]

Applying the definition of expected value and switching the order of summation,

\displaystyle  \begin{aligned} \mathbf E_{x_i}\left[ p_i(x_i) \right] &= p_i(0) + \int_{t=0}^{1000} t \cdot q_i(t) \; d\mu_i - \int_{x=0}^{1000} \int_0^{x} q(t) \; dt d\mu_i \\ &= \int_{t=0}^{1000} t \cdot q_i(t) \; d\mu_i - \int_{t=0}^{1000} q(t) \int_{x=t}^{1000} \int_0^{x} d\mu_i dt \\ &= \int_{t=0}^{1000} t \cdot q_i(t) \; d\mu_i - \int_{t=0}^{1000} q_i(t) \cdot \mu_i([t, 1000]) dt \\ &= \int_{t=0}^{1000} t \cdot q_i(t) \; d\mu_i - \int_{t=0}^{1000} q_i(t) \cdot \frac{1-\mu_i([0, t])} {\frac{d\mu_i}{dt}(t)} d\mu_i \\ &= \int_{t=0}^{1000} q_i(t) \left( t - \frac{1-\mu_i([0,t])} {\frac{d\mu_i}{dt}(t)} \right) d\mu_i \end{aligned}

Thus, we define for the player {i} their virtual valuation to be the thing that we just obtained in this computation:

\displaystyle  \psi_i(t) = t - \frac{1-\mu_i([0,t])}{\frac{d\mu_i}{dt}(t)}.


\displaystyle  \mathbf E_{x_i}\left[ p_i(x_i) \right] = \int_{t=0}^{1000} q_i(t) \cdot \psi_i(t) \; dt.  \ \ \ \ \ (1)

The virtual valuation {\psi_i(t)} can be thought of as the expected value of the amount of money we extract if we give the object to player {i} when she has type {t}. Note it has the very important property of not depending on {q_i}.

Now, let’s observe that

\displaystyle  \begin{aligned} \sum_{i=1}^N \mathbf E_{x_i} \left[ p_i(x_i) \right] &= \sum_{i=1}^N \int_{t=0}^{1000} q_i(t) \psi_i(t) \; d\mu_i \\ &= \sum_{i=1}^N \int_{x_1=0}^{1000} \dots \int_{x_N=0}^{1000} Q_i(\vec x) \psi(x_i) \; d\mu_1 d\mu_2 \dots d\mu_N \\ &= \int_{x_1=0}^{1000} \dots \int_{x_N=0}^{1000} \sum_{i=1}^N \left( Q_i(\vec x) \psi(x_i) \right) \; d\mu_1 d\mu_2 \dots d\mu_N. \end{aligned}

Now, our goal is to select the function {Q} to maximize this. Consider a particular point {\vec x = (x_1, \dots, x_N)}. Then we’re trying to pick {Q_1(\vec x)}, {Q_2(\vec x)}, \dots, {Q_n(\vec x)} to maximize the value of

\displaystyle  Q_1(\vec x) \psi_1(x_1) + Q_2(\vec x) \psi_2(x_2) + \dots + Q_n(\vec x) \psi_n(x_n)

subject to {\sum Q_i(\vec x) \le 1}. The {\psi_i}‘s here depend only on the distribution {\mu}. This is thus a convex optimization problem, so the solution is obvious: put all the weight on the biggest positive {\psi_i(x_i)}, breaking ties arbitrarily. In other words, if {k = \text{argmax }_i \psi_i(x_i)} and {\psi_k(x_k) > 0}, then set {Q_k(\vec x) = 1} and all the other coefficients to be zero. (Ties broken arbitrarily as usual.) To be explicit, we set

\displaystyle  Q_k(\vec x) = \begin{cases} 1 & \psi_k(x_k) \ge 0 \text{ is maximal (ties broken arbitrarily)} \\ 0 & \text{else}. \end{cases}

So we should think of this as a second-price auction with discriminatory reserve prices. Moreover, the “second-price” payment is done based on {\psi_i}, so rather than “pay second highest bid” we instead have “pay smallest amount needed to win”, i.e. the smallest {y} such that {\psi_k(y) \ge \psi_j(x_j)} for all {j \neq k}.

Unfortunately, since the {\mu_i} are arbitrary the resulting {Q_i} might be strange enough that the game fails to have a reasonable strategy {\sigma}; in other words we’re worried the maximum might not be achievable. The easiest thing to do is write down a condition to handle this:

Definition 18

We say {\mu = \{\mu_i\}_{i=1}^N} is regular if the virtual valuations {\psi_i : [0,1000] \rightarrow \mathbb R} are strictly increasing for all {i}.

Theorem 19 (Regularity implies optimal auction is achievable)

Assume regularity of {\mu}. Consider a second-price auction with discriminatory reserve prices: the reserve price for player {i} is the smallest {x} such that {\psi_i(x) > 0}, and the winner {k} pays the smallest amount needed to win.

This is an incentive compatible mechanism which maximizes the expected revenue of the auctioneer.

Proof: The {Q} described in the theorem is the one we mentioned earlier. The hypothesis defines {P} as follows:

  • If {\psi_k(x_k) > 0} is maximal, then that player wins and pays the smallest amount {y} such that {\psi_k(y)} still exceeds all other {\psi_j(x_j)}.
  • Otherwise, the item is not sold.

The fact that this mechanism {M = (P,Q,\mu)} is incentive compatible is more or less the same as before (bidding truthfully is a weakly dominant strategy). Moreover we already argued above that this allocation {Q} maximizes revenue. \Box

You can and should think of this as a “reserve price second price auction” except with virtual valuations instead. The winner is the player with the highest virtual valuation, who is then allowed to retroactively change their bid so long as they still win.

To see this in action:

Corollary 20 (Optimal symmetric uniform auction)

Consider an auction in which {n} players have uniform values in {[0,1000]}. Then the optimal auction is a second-price auction with reserve price {\$500}. The expected earning of the auctioneer is

\displaystyle  1000 \cdot \frac{n - 1 + \left( \frac{1}{2} \right)^n}{n+1}.

Proof: We compute each {\psi_i} as

\displaystyle  \psi_i(x) = x - \frac{1 - \mu_i([0,x])}{\frac{d\mu_i}{dt}(x)} = x - \frac{1-x/1000}{1/1000} = 2x - 1000.

Hence, we usually want to award the item to the player with the largest virtual valuation (i.e. the highest bidder), setting the reserve price at {500} for everyone since {\psi_i(500) = 0}. By (1) the expected payment from a player equals

\displaystyle  \frac{1}{1000} \int_{x=0}^{1000} \psi_i(x) \cdot q_i(x) \; dx = \frac{1}{1000}\int_{x=500}^{1000} (2x-1000) \cdot (x/1000)^{n-1} \; dx

Simplifying and multiplying by {n} gives the answer. \Box

More generally, in an asymmetric situation the optimal reserve prices are discriminatory and vary from player to player. As a nice exercise to get used to this:

Exercise 21

Find the optimal auction mechanism if two players are bidding, with one player having value distributed uniformly in {[0,500]} and the other player having value distributed uniformly in {[0,1000]}.

8. Against revelation

Here’s a pedagogical point: almost all sources state early on the so-called “revelation principle”.

Proposition 22 (Revelation principle)

Let {M = (M, \sigma)} be a direct mechanism. Then there exists a incentive compatible mechanism {M^\ast} such that

  • {M^\ast = (M^\ast, \mathrm{id})} is incentive compatible, and
  • For any {i}, {p_i^\ast = p_i}, {q_i^\ast = q_i}, {u_i^\ast = u_i}, i.e. at the equilibrium point, the expected payoffs and allocations don’t change.

Proof: Consider {M = (M, \sigma)}. Then {M^\ast} is played as follows:

  • Each player {i} has an advisor to whom it tells their value {x_i}
  • The advisor and figures out the optimal bid {\sigma_i(x_i)} in {M}, and submits this bid on the player’s behalf.
  • The game {M} is played with the advisor’s bids, and the payments / allocations are given to corresponding players.

Clearly the player should be truthful to their advisor. \Box

Most sources go on to say that this makes their life easier, because now they can instead say “it suffices to study incentive compatible mechanisms”. For example, one can use this to

  • Replace “direct mechanism” with “incentive-compatible direct mechanism”.
  • Replace “there exists a direct mechanism {(M, \sigma)}” with “there exists a {(P,Q)} which is incentive compatible”.

However, I personally think this is awful. Here is why.

The proof of Revelation is almost tautological. Philosophically, it says that you should define the functions {q_i}, {p_i}, {u_i} in terms of the equilibrium {\sigma}. Authors which restrict to incentive compatible mechanisms are hiding this fact behind Revelation: now to the reader it’s no longer clear whether {u_i} should be interpreted as taking a bid or value as input.

Put another way, the concept of a bid/value should be kept segregated. That’s why I use {x_i} only for values, {b_i} only for bids, and build in the equilibrium {\sigma} as into {u_i}, {p_i}, {q_i}; So {u_i(x)} is the expected utility of bidding {\sigma(x_i)} in the equilibrium. Revelation does the exact opposite: it lets authors promiscuously mix the concepts of values and bids, and pushes {\sigma} out of the picture altogether.

Writing Olympiad Geometry Problems

You can use a wide range of wild, cultivated or supermarket greens in this recipe. Consider nettles, beet tops, turnip tops, spinach, or watercress in place of chard. The combination is also up to you so choose the ones you like most.

— Y. Ottolenghi. Plenty More

In this post I’ll describe how I come up with geometry proposals for olympiad-style contests. In particular, I’ll go into detail about how I created the following two problems, which were the first olympiad problems which I got onto a contest. Note that I don’t claim this is the only way to write such problems, it just happens to be the approach I use, and has consistently gotten me reasonably good results.

[USA December TST for 56th IMO] Let {ABC} be a triangle with incenter {I} whose incircle is tangent to {\overline{BC}}, {\overline{CA}}, {\overline{AB}} at {D}, {E}, {F}, respectively. Denote by {M} the midpoint of {\overline{BC}} and let {P} be a point in the interior of {\triangle ABC} so that {MD = MP} and {\angle PAB = \angle PAC}. Let {Q} be a point on the incircle such that {\angle AQD = 90^{\circ}}. Prove that either {\angle PQE = 90^{\circ}} or {\angle PQF = 90^{\circ}}.


[Taiwan TST Quiz for 56th IMO] In scalene triangle {ABC} with incenter {I}, the incircle is tangent to sides {CA} and {AB} at points {E} and {F}. The tangents to the circumcircle of {\triangle AEF} at {E} and {F} meet at {S}. Lines {EF} and {BC} intersect at {T}. Prove that the circle with diameter {ST} is orthogonal to the nine-point circle of {\triangle BIC}.


1. General Procedure

Here are the main ingredients you’ll need.

  • The ability to consistently solve medium to hard olympiad geometry problems. The intuition you have from being a contestant proves valuable when you go about looking for things.
  • In particular, a good eye: in an accurate diagram, you should be able to notice if three points look collinear or if four points are concyclic, and so on. Fortunately, this is something you’ll hopefully have just from having done enough olympiad problems.
  • Geogebra, or some other software that will let you quickly draw and edit diagrams.

With that in mind, here’s the gist of what you do.

  1. Start with a configuration of your choice; something that has a bit of nontrivial structure in it, and add something more to it. For example, you might draw a triangle with its incircle and then add in the excircle tangency point, and the circle centered at {BC} passing through both points (taking advantage of the fact that the two tangency points are equidistant from {B} and {C}).
  2. Start playing around, adding in points and so on to see if anything interesting happens. You might be guided by some actual geometry constructions: for example, if you know that the starting configuration has a harmonic bundle in it, you might project this bundle to obtain the new points to play with.
  3. Keep going with this until you find something unexpected: three points are collinear, four points are cyclic, or so on. Perturb the diagram to make sure your conjecture looks like it’s true in all cases.
  4. Figure out why this coincidence happened. This will probably add more points to you figure, since you often need to construct more auxiliary points to prove the conjecture that you have found.
  5. Repeat the previous two steps to your satisfaction.
  6. Once you are happy with what you have, you have a nontrivial statement and probably several things that are equivalent to it. Pick the one that is most elegant (or hardest), and erase auxiliary points you added that are not needed for the problem statement.
  7. Look for other ways to reduce the number of points even further, by finding other equivalent formulations that have fewer points.

Or shorter yet: build up, then tear down.

None of this makes sense written this abstractly, so now let me walk you through the two problems I wrote.

2. The December TST Problem

In this narrative, the point names might be a little strange at first, because (to make the story follow-able) I used the point names that ended up in the final problem, rather than ones I initially gave. Please bear with me!

I began by drawing a triangle {ABC} (always a good start\dots) and its incircle, tangent to side {BC} at {D}. Then, I added in the excircle touch point {T}, and drew in the circle with diameter {DT}, which was centered at the midpoint {M}. This was a coy way of using the fact that {MD = MT}; I wanted to see whether it would give me anything interesting.

So, I now had the following picture.


Now I had two circles intersecting at a single point {D}, so I added in {Q}, the second intersection. But really, this point {Q} can be thought of another way. If we let {DS} be the diameter of the incircle, then as {DT} is the other diameter, {Q} is actually just the foot of the altitude from {D} to line {ST}.

But recall that {A}, {S}, {T} are collinear! (Again, this is why it’s helpful to be familiar with “standard” contest configurations; you see these kind of things immediately.) So {Q} in fact lies on line {AT}.


This was pretty cool, though not yet interesting enough to be a contest problem. So I looked for most things that might be true.

I don’t remember what I tried next; it didn’t do anything interesting. But I do remember the thing I tried after that: I drew in the angle bisector, line {AI}. And then, I noticed a big coincidence: the first intersection of {AI} with the circle with diameter {DT} seemed to lie on line {DE}! I was initially confused by this; it didn’t seem like it could possibly be true due to symmetry reasons. But in my diagram, it was indeed correct. A moment later, I realized the reason why this was plausible: in fact, the second intersection of line {AI} with the circle was on line {DF}.


Now, I could not see quickly at all why this was true. So I started trying to prove it, but initially failed: however, I managed to show (via angle chasing) that

\displaystyle D, P, E \text{ collinear} \iff \angle PQE = 90^\circ.

So, at least I had an interesting equivalent statement.


After another half hour of trying to prove my conjecture, I finally realized what was happening. The point {P} was the one attached to a particular lemma: the {A}-bisector, {B}-midline, and {C} touch-chord are concurrent, and from this {MD = MP} just follows by some similar triangles. So, drawing in the point {N} (the midpoint of {AB}), I had the full configuration which gave the answer to my conjecture.


Finally, I had to clean up the mess that I had made. How could I do this? Well, the points {N}, {S} could be eliminated easily enough. And we could re-define {Q} to be a point on the incircle such that {\angle AQD = 90^\circ}. This actually eliminated the green circle and point {T} altogether, provided we defined {P} by just saying that it was on the angle bisector, and that {MD = MP}. (So while the circle was still implicit in the condition {MD = MP}, it was no longer explicitly part of the problem.)

Finally, we could even remove the line through {D}, {P} and {E}; we ask the contestant to prove {\angle PQE = 90^\circ}.


And that was it!

3. The Taiwan TST Problem

In fact, the starting point of this problem was the same lemma which provided the key to the previous solution: the circle with diameter {BC} intersects the {B} and {C} bisectors on the {A} touch chord. Thus, we had the following diagram.


The main idea I had was to look at the points {D}, {X}, {Y} in conjunction with each other. Specifically, this was the orthic triangle of {\triangle BIC}, a situation which I had remembered from working on Iran TST 2009, Problem 9. So, I decided to see what would happen if I drew in the nine-point circle of {\triangle BIC}. Naturally, this induces the midpoint {M} of {BC}.


At this point, notice (or recall!) that line {AM} is concurrent with lines {DI} and {EF}.


So the nine-point circle of the problem is very tied down to the triangle {BIC}. Now, since I was in the mood for something projective, I constructed the point {T}, the intersection of lines {EF} and {BC}. In fact, what I was trying to do was take perspectivity through {I}. From this we actually deduce that {(T,K;X,Y)} is a harmonic bundle.


Now, what could I do with this picture? I played around looking for some coincidences, but none immediately presented themselves. But I was enticed by the point {T}, which was somehow related to the cyclic complete quadrilateral {XYMD}. So, I went ahead and constructed the pole of {T} to the nine-point circle, letting it hit line {BC} at {L}. This was aimed at “completing” the picture of a cyclic quadrilateral and the pole of an intersection of two sides. In particular, {(T,L;D,M)} was harmonic too.


I spent a long time thinking about how I could make this into a problem. I unfortunately don’t remember exactly what things I tried, other than the fact that I was taking a lot of perspectivity. In particular, the “busiest” point in the picture is {K}, so it makes sense to try and take perspectives through it. Especially enticing was the harmonic bundle

\displaystyle \left( \overline{KT}, \overline{KL}; \overline{KD}, \overline{KM} \right) = -1.

How could I use this to get a nice result?

Finally about half an hour I got the right idea. We could take this bundle and intersect it with the ray {AI}! Now, letting {N} be the midpoint {EF}, we find that three of the points in the harmonic bundle we obtain are {A}, {I}, and {N}; let {S} be the fourth point, which is the intersection of line {KL} with {AI}. Then by hypothesis, we ought to have {(A,I;N,S) = -1}. But from this we know exactly what the point {S}. Just look at the circumcircle of triangle {AEF}: as this has diameter {AI}, we see that {S} is the intersection of the tangents at {E} and {F}.


Consequently, we know that the point {S}, defined very naturally in terms of the original picture, lies on the polar of {T} to the nine-point circle. By simply asking the contestant to prove this, we thus eliminate all the points {K}, {M}, {D}, {N}, {I}, {X}, and {Y} completely from the picture, leaving only the nine-point circle. Finally, instead of directly asking the contestant to show that {T} lies on the polar of {S}, one can rephrase the problem as saying “the circle with diameter {ST} is orthogonal to the nine-point circle of {\triangle BIC}”, concealing all the work that went into the creation of the problem.



The Mixtilinear Incircle

This blog post corresponds to my newest olympiad handout on mixtilinear incircles.

My favorite circle associated to a triangle is the {A}-mixtilinear incircle. While it rarely shows up on olympiads, it is one of the richest configurations I have seen, with many unexpected coincidences showing up, and I would be overjoyed if they become fashionable within the coming years.

Here’s the picture:


The points {D} and {E} are the contact points of the incircle and {A}-excircle on the side {BC}. Points {M_A}, {M_B}, {M_C} are the midpoints of the arcs.

As a challenge to my recent USAMO class (I taught at A* Summer Camp this year), I asked them to find as many “coincidences” in the picture as I could (just to illustrate the richness of the configuration). I invite you to do the same with the picture above.

The results of this exercise were somewhat surprising. Firstly, I found out that students without significant olympiad experience can’t “see” cyclic quadrilaterals in a picture. Through lots of training I’ve gained the ability to notice, with some accuracy, when four points in a diagram are concyclic. This has taken me a long way both in setting problems and solving them. (Aside: I wonder if it might be possible to train this skill by e.g. designing an “eyeballing” game with real olympiad problems. I would totally like to make this happen.)

The other two things that happened: one, I discovered one new property while preparing the handout, and two, a student found yet another property which I hadn’t known to be true before. In any case, I ended up covering the board in plenty of ink.


Here’s the list of properties I have.

  1. First, the classic: by Pascal’s Theorem on {TM_CCABM_B}, we find that points {B_1}, {I}, {C} are collinear; hence the contact chord of the {A}-mixtilinear incircle passes through the incenter. The special case of this problem with {AB = AC} appeared in IMO 1978.
    • Then, by Pascal on {BCM_CTM_AA}, we discover that lines {BC}, {B_1C_1}, and {TM_A} are also concurrent.
    • This also lets us establish (by angle chasing) that {BB_1IT} and {CC_1IT} are concyclic. In addition, lines {BM_B} and {CM_C} are tangents to these circumcircles at {I} (again by angle chasing).
  2. An Iran 2002 problem asks to show that ray {TI} passes through the point diametrically opposite {M_A} on the circumcircle. This is solved by noticing that {TA} is a symmedian of the triangle {TB_1C_1} and (by the previous fact) that {TI} is a median. This is the key lemma in Taiwan TST 2014, Problem 3, which is one of my favorite problems (a nice result by Cosmin Pohoatza).
  3. Lines {AT} and {AE} are isogonal. This was essentially EGMO 2012, Problem 5, and the “morally correct” solution is to do an inversion at {A} followed by a reflection along the {\angle A}-bisector (sometimes we call this a “{\sqrt{bc}} inversion”).
    • As a consequence of this, one can also show that lines {TA} and {TD} are isogonal (with respect to {\angle BTC}).
    • One can also deduce from this that the circumcircle of {\triangle TDM_A} passes through the intersection of {BC} and {AM_A}.
  4. Lines {AD} and {TM_A} meet on the mixtilinear incircle. (Homothety!) 
  5. Moreover, line {AT} passes through the exsimilicenter of the incircle and circumcircle, by, say Monge d’Alembert. Said another way, the mentioned exsimilicenter is the isogonal conjugate of the Nagel point.

To put that all into one picture:


Cauchy’s Functional Equation and Zorn’s Lemma

This is a draft of an appendix chapter for my Napkin project.

In the world of olympiad math, there’s a famous functional equation that goes as follows:

\displaystyle f : {\mathbb R} \rightarrow {\mathbb R} \qquad f(x+y) = f(x) + f(y).

Everyone knows what its solutions are! There’s an obvious family of solutions {f(x) = cx}. Then there’s also this family of… uh… noncontinuous solutions (mumble grumble) pathological (mumble mumble) Axiom of Choice (grumble).

Don’t worry, I know what I’m doing!

There’s also this thing called Zorn’s Lemma. It sounds terrifying, because it’s equivalent to the Axiom of Choice, which is also terrifying because why not.

In this post I will try to de-terrify these things, because they’re really not terrifying and I’m not sure why no one bothered to explain this properly yet. I have yet to see an olympiad handout that explains how you would construct a pathological solution, even though it’s really quite natural. So let me fix this problem now…

1.Let’s Construct a Monster

Let’s try to construct a “bad” function {f} and see what happens.

By scaling, let’s assume WLOG that {f(1) = 1}. Thus {f(n) = n} for every integer {n}, and you can easily show from here that

\displaystyle f\left( \frac mn \right) = \frac mn.

So {f} is determined for all rationals. And then you get stuck.

None of this is useful for determining, say, {f(\sqrt 2)}. You could add and subtract rational numbers all day and, say, {\sqrt 2} isn’t going to show up at all.

Well, we’re trying to set things on fire anyways, so let’s set

\displaystyle f(\sqrt 2) = 2015

because why not? By the same induction, we get {f(n\sqrt2) = 2015n}, and then that

\displaystyle f\left( a + b \sqrt 2 \right) = a + 2015b.

Here {a} and {b} are rationals. Well, so far so good — as written, this is a perfectly good solution, other than the fact that we’ve only defined {f} on a tiny portion of the real numbers.

Well, we can do this all day:

\displaystyle f\left( a + b \sqrt 2 + c \sqrt 3 + d \pi \right) = a + 2015b + 1337c - 999d.

Perfectly consistent.

You can kind of see how we should keep going now. Just keep throwing in new real numbers which are “independent” to the previous few, assigning them to whatever junk we want. It feels like it should be workable. . .

In a moment I’ll explain what “independent” means (though you might be able to guess already), but at the moment there’s a bigger issue: no matter how many numbers we throw, it seems like we’ll never finish. Let’s address the second issue first.

2. Review of Finite Induction

When you do induction, you get to count off {1}, {2}, {3}, … and so on. So for example, suppose we had a “problem” such as the following: Prove that the intersection of {n} open intervals is either {\varnothing} or an open interval. You can do this by induction easily: it’s true for {n = 2}, and for the larger cases it’s similarly easy.

But you can’t conclude from this that infinitely many open intervals intersect at some open interval. Indeed, this is false: consider the intervals

\displaystyle \left( -1, 1 \right), \quad \left( -\frac12, \frac12 \right), \quad \left( -\frac13, \frac13 \right), \quad \left( -\frac14, \frac14 \right), \quad \dots

This infinite set of intervals intersects at a single point {\{0\}}!

The moral of the story is that induction doesn’t let us reach infinity. Too bad, because we’d have loved to use induction to help us construct a monster. That’s what we’re doing, after all — adding things in one by one.

3. Transfinite Induction

Well, it turns out we can, but we need a new notion of number.

For this we need a notion of an ordinal number. I defined these in their full glory a previous blog post, but I don’t need the full details of that. Here’s what I want to say: after all the natural numbers

\displaystyle 0, \; 1, \; \dots,

I’ll put a new number called {\omega}, representing how large the natural numbers are. After that there’s a number called

\displaystyle \omega+1, \; \omega+2, \; \dots

and eventually a number called {2\omega}.

The list goes on:

\displaystyle \begin{aligned} 0, & 1, 2, 3, \dots, \omega \\ & \omega+1, \omega+2, \dots, \omega+\omega \\ & 2\omega+1, 2\omega+2, \dots, 3\omega \\ & \vdots \\ & \omega^2 + 1, \omega^2+2, \dots \\ & \vdots \\ & \omega^3, \dots, \omega^4, \dots, \omega^\omega \\ & \vdots, \\ & \omega^{\omega^{\omega^{\dots}}} \\ \end{aligned}

Pictorially, it kind of looks like this:


Anyways, in the same way that natural numbers dominate all finite sets, the ordinals dominate all the sets.

Theorem 1 For every set {S} there’s some ordinal {\alpha} which is bigger than it.

But it turns out (and you can intuitively see) that as large as the ordinals grow, there is no infinite descending chain. Meaning: if I start at an ordinal (like {2 \omega + 4}) and jump down, I can only take finitely many jumps before I hit {0}. (To see this, try writing down a chain starting at {2 \omega + 4} yourself.) Hence, induction and recursion still work verbatim:

Theorem 2 Given a statement {P(-)}, suppose that

  • {P(0)} is true, and
  • If {P(\alpha)} is true for all {\alpha < \beta}, then {P(\beta)} is true.

Then {P(\beta)} is true.

Similarly, you’re allowed to do recursion to define {x_\beta} if you know the value of {x_\alpha} for all {\alpha < \beta}.

The difference from normal induction or recursion is that we’ll often only do things like “define {x_{n+1} = \dots}”. But this is not enough to define {x_\alpha} for all {\alpha}. To see this, try using our normal induction and see how far we can climb up the ladder.

Answer: you can’t get {\omega}! It’s not of the form {n+1} for any of our natural numbers {n} — our finite induction only lets us get up to the ordinals less than {\omega}. Similarly, the simple {+1} doesn’t let us hit the ordinal {2\omega}, even if we already have {\omega+n} for all {n}. Such ordinals are called limit ordinals. The ordinal that are of the form {\alpha+1} are called successor ordinals.

So a transfinite induction or recursion is very often broken up into three cases. In the induction phrasing, it looks like

  • (Zero Case) First, resolve {P(0)}.
  • (Successor Case) Show that from {P(\alpha)} we can get {P(\alpha+1)}.
  • (Limit Case) Show that {P(\lambda)} holds given {P(\alpha)} for all {\alpha < \lambda}, where {\lambda} is a limit ordinal.

Similarly, transfinite recursion often is split into cases too.

  • (Zero Case) First, define {x_0}.
  • (Successor Case) Define {x_{\alpha+1}} from {x_\alpha}.
  • (Limit Case) Define {x_\lambda} from {x_\alpha} for all {\alpha < \lambda}, where {\lambda} is a limit ordinal.

In both situations, finite induction only does the first two cases, but if we’re able to do the third case we can climb far above the barrier {\omega}.

4. Wrapping Up Functional Equations

Let’s return to solving our problem.

Let {S_n} denote the set of “base” numbers we have at the {n} the step. In our example, we might have

\displaystyle S_1 = \left\{ 1 \right\}, \quad S_2 = \left\{ 1, \sqrt 2 \right\}, \quad S_3 = \left\{ 1, \sqrt 2, \sqrt 3 \right\}, \quad S_4 = \left\{ 1, \sqrt 2, \sqrt 3, \pi \right\}, \quad \dots

and we’d like to keep building up {S_i} until we can express all real numbers. For completeness, let me declare {S_0 = \varnothing}.

First, I need to be more precise about “independent”. Intuitively, this construction is working because

\displaystyle a + b \sqrt 2 + c \sqrt 3 + d \pi

is never going to equal zero for rational numbers {a}, {b}, {c}, {d} (other than all zeros). In general, a set {X} of numbers is “independent” if the combination

\displaystyle c_1x_1 + c_2x_2 + \dots + c_mx_m = 0

never occurs for rational numbers {{\mathbb Q}} unless {c_1 = c_2 = \dots = c_m = 0}. Here {x_i \in X} are distinct. Note that even if {X} is infinite, I can only take finite sums! (This notion has a name: we want {X} to be linearly independent over {{\mathbb Q}}.)

When do we stop? We’d like to stop when we have a set {S_{\text{something}}} that’s so big, every real number can be written in terms of the independent numbers. (This notion also has a name: it’s called a {{\mathbb Q}}-basis.) Let’s call such a set spanning; we stop once we hit a spanning set.

The idea that we can induct still seems okay: suppose {S_\alpha} isn’t spanning. Then there’s some number that is independent of {S_\alpha}, say {\sqrt{2015}\pi} or something. Then we just add it to get {S_{\alpha+1}}. And we keep going.

Unfortunately, as I said before it’s not enough to be able to go from {S_\alpha} to {S_{\alpha+1}} (successor case); we need to handle the limit case as well. But it turns out there’s a trick we can do. Suppose we’ve constructed all the sets {S_0}, {S_1}, {S_2}, …, one for each positive integer {n}, and none of them are spanning. The next thing I want to construct is {S_\omega}; somehow I have to “jump”. To do this, I now take the infinite union

\displaystyle S_\omega \overset{\text{def}}{=} S_0 \cup S_1 \cup S_2 \cup \dots.

The elements of this set are also independent (why?).

Ta-da! With the simple trick of “union all the existing sets”, we’ve just jumped the hurdle to the first limit ordinal {\omega}. Then we can construct {S_{\omega+1}}, {S_{\omega+2}}, …, and for the next limit we just do the same trick of “union-ing” all the previous sets.

So we can formalize the process as follows:

  1. Let {S_0 = \varnothing}.
  2. For a successor stage {S_{\alpha+1}}, add any element to {S_\alpha} to obtain {S_{\alpha+1}}.
  3. For a limit stage {S_{\lambda}}, take the union {\bigcup_{\gamma < \lambda} S_\gamma}.

How do we know that we’ll stop eventually? Well, the thing is that this process consumes a lot of real numbers. In particular, the ordinals get larger than the size of {{\mathbb R}}. Hence if we don’t stop we will quite literally reach a point where we have used up every single real number. Clearly that’s impossible, because by then the elements can’t possibly be independent! (EDIT Dec 20 2015: To be clear, the claim that “ordinals get larger than the size of the reals” requires the Axiom of Choice; one can’t do this construction using transfinite induction alone. Thanks reddit for calling me out on this.)

So by transfinite recursion (and Choice), we eventually hit some {S_\gamma} which is spanning: the elements are all independent, but every real number can be expressed using it. Done! This set has a name: a Hamel basis.

Thus this problem is dead, dead, dead. Any questions?


5. Zorn’s Lemma

Now I can tell you what Zorn’s Lemma is: it lets us do the same thing in any poset.

We can think of the above example as follows: consider all sets of independent elements. These form a partially ordered set by inclusion, and what we did was quite literally climb up a chain

\displaystyle S_0 \subset S_1 \subset S_2 \subset \dots.

It’s not quite climbing since we weren’t just going one step at a time: we had to do “jumps” to get up to {S_\omega} and resume climbing. But the main idea is to climb up a poset until we’re at the very top; in the previous case, when we reached the spanning set.

The same thing works verbatim with any partially ordered set {\mathcal P}. Let’s define some terminology. A local maximum (or maximal element) of the entire poset {\mathcal P} is an element which has no other elements strictly greater than it.

Now a chain of length {\gamma} is a set of elements {p_\alpha} for every {\alpha < \gamma} such that {p_0 < p_1 < p_2 < \dots}. (Observe that a chain has a last element if and only if {\gamma} is a successor ordinal, like {\omega+3}.) An upper bound to a chain is an element {\tilde p} which is greater than or equal to all elements of the chain. In particular, if {\gamma} is a successor ordinal, then just taking the last element of the chain works.

In this language, Zorn’s Lemma states that

Theorem 3 (Zorn’s Lemma) Let {\mathcal P} be a nonempty partially ordered set. If every chain has an upper bound, then {\mathcal P} has a local maximum.

Chains with length equal to a successor ordinal always have upper bounds, but this is not true in the limit case. So the hypothesis of Zorn’s Lemma is exactly what lets us “jump” up to define {p_\omega} and other limit ordinals. And the proof of Zorn’s Lemma is straightforward: keep climbing up the poset at successor stages, using Zorn’s condition to jump up at limit stages, and thus building a really long chain. But we have to eventually stop, or we literally run out of elements of {\mathcal P}. And the only possible stopping point is a local maximum.

If we want to phrase our previous solution in terms of Zorn’s Lemma, we’d say: Proof: Look at the poset whose elements are sets of independent real numbers. Every chain {S_0 \subset S_1 \subset \dots} has an upper bound {\bigcup S_\alpha} (which you have to check is actually an element of the poset). Thus by Zorn, there is a local maximum {S}. Then {S} must be spanning, because otherwise we could add an element to it. \Box

So really, Zorn’s Lemma is encoding all of the work of climbing that I argued earlier. It’s a neat little package that captures all the boilerplate, and tells you exactly what you need to check.

One last thing you might ask: where is the Axiom of Choice used? Well, the idea is that for any chain there could be lots of {\tilde p}‘s, and you need to pick one of them. Since you are making arbitrary choices infinitely many times, you need the Axiom of Choice. But really, it’s nothing special. [EDIT: AM points out that in order to talk about cardinalities in Theorem 1, one also needs the Axiom of Choice.]

6. Conclusion

In the words of Timothy Gowers,

If you are building a mathematical object in stages and find that (i) you have not finished even after infinitely many stages, and (ii) there seems to be nothing to stop you continuing to build, then Zorn’s lemma may well be able to help you.

Really, there’s nothing tricky at all here. People seem scared of Zorn’s Lemma, and claim it’s not intuitive or something. But really, all we’re doing is climbing up a poset. Nothing tricky at all.

Three Properties of Isogonal Conjugates

In this post I’ll cover three properties of isogonal conjugates which were only recently made known to me. These properties are generalization of some well-known lemmas, such as the incenter/excenter lemma and the nine-point circle.

1. Definitions

Let {ABC} be a triangle with incenter {I}, and let {P} be any point in the interior of {ABC}. Then we obtain three lines {AP}, {BP}, {CP}. Then the reflections of these lines across lines {AI}, {BI}, {CI} always concur at a point {Q} which is called the isogonal conjugate of {P}. (The proof of this concurrence follows from readily from Trig Ceva.) When {P} lies inside {ABC}, then {Q} is the point for which {\angle BAP = \angle CAQ} and so on.


The isogonal conjugate of {P} is sometimes denoted {P^\ast}. Note that {(P^\ast)^\ast = P}.

Examples of pairs of isogonal conjugates include the following.

  1. The incenter is its own isogonal conjugate. Similarly, each excenter is also its own isogonal conjugate.
  2. The isogonal conjugate of the circumcenter is the orthocenter.
  3. The isogonal conjugate of the centroid is the symmedian point.
  4. The isogonal conjugate of the Nagel point is the point of concurrence of {AT_A}, {BT_B}, {CT_C}, where {T_A} is the contact point of the {A}mixtilinear incircle. The proof of this result was essentially given as Problem 5 of the European Girl’s Math Olympiad.

2. Inverses and Circumcircles

You may already be aware of the famous result (which I always affectionately call “Fact 5”) that the circumcenter of {BIC} is the midpoint of arc {BC} of the circumcircle of {ABC}. Indeed, so is the circumcenter of triangle {BI_AC}, where {I_A} is the {A}-excenter.


In fact, it turns out that we can generalize this result for arbitrary isogonal conjugates as follows.

Theorem 1 Let {P} and {Q} be isogonal conjugates. Then the circumcenters of {\triangle BPC} and {\triangle BQC} are inverses with respect to the circumcircle of {\triangle ABC}.


Proof: This is just angle chasing. Let {O_P} and {O_Q} be the desired circumcenters. It’s clear that both {O_P} and {O_Q} lie on the perpendicular bisector of {\overline{BC}}. Angle chasing allows us to compute that

\displaystyle \angle BO_PO = \frac 12 \angle BO_PC = 180^{\circ} - \angle BPC.

Similarly, {\angle BO_QO = 180^{\circ} - \angle BQC}. But the reader can check that {\angle BPC + \angle BQC = 180^{\circ} + A}. Using this we can show that {\angle OBO_Q = \angle BO_PO}, so {\triangle OBO_P \sim \triangle OO_QB}, as needed. \Box

When we take {P} and {Q} to be {I} (or {I_A}), we recover the Fact 5 we mentioned above. When we take {P} to be the orthocenter and {Q} to be the circumcenter, we find that the circumcenter of {BHC} is the inverse of the circumcenter of {BOC}. But the inverse of the circumcenter of {BOC} is the reflection of {O} over {\overline{BC}}. Thus we derive that {\triangle BHC} and {\triangle BOC} have circumcircles which are just reflections over {\overline{BC}}.

3. Pedal Circles

You may already be aware of the nine-point circle, which passes through the midpoints and feet of the altitudes of {ABC}. In fact, we can obtain such a circle for any pair of isogonal conjugates.

Theorem 2 Let {P} and {Q} be isogonal conjugates in the interior of {\triangle ABC}. The pedal triangles of {P} and {Q} share a circumcircle. Moreover, the center of this circle is the midpoint {M} of {\overline{PQ}}.


Upon taking {P=H} and {Q=O} we recover the nine-point circle. Of course, the incircle is the special case {P=Q=I}!

Proof: Let {\triangle P_AP_BP_C} and {\triangle Q_AQ_BQ_C} be the pedal triangles. We leave the reader to check that

\displaystyle AP_C \cdot AQ_C = AP \cdot AQ \cdot \cos \angle BAP \cdot \cos \angle BAQ = AP_B \cdot AQ_B.

Consequently, the points {P_C}, {Q_C}, {P_B}, {Q_B} are concyclic. The circumcenter of these four points is the intersection of the perpendicular bisectors of segments {\overline{P_CQ_C}} and {\overline{P_BQ_B}}, which is precisely {M}. Thus

\displaystyle MP_C = MQ_C = MP_B = MQ_B.

Similarly work with the other vertices shows that {M} is indeed the desired circumcenter. \Box

There is a second way to phrase this theorem by taking a homothety at {Q}.

Corollary If the point {Q} is reflected about the sides {\overline{AB}}, {\overline{BC}}, and {\overline{CA}}, then the resulting triangle has circumcenter {P}.

4. Ellipses

We can actually derive the following remarkable result from the above theorem.

Theorem 3 An ellipse {\mathcal E} is inscribed in triangle {ABC}. Then the foci {P} and {Q} are isogonal conjugates.

Of course, the incircle is just the special case when the ellipse is a circle.


Proof: We will deduce this from the corollary. Let the ellipse be tangent at points {D}, {E}, {F}. Moreover, let the reflection of {Q} about the sides of {\triangle ABC} be points {X}, {Y}, {Z}. By definition, there is a common sum {s} with

\displaystyle s = PD + DQ = PE + EQ + PF + FQ.

Because of the tangency condition, the points {P}, {D}, {X} are collinear. But now

\displaystyle PX = PD+DX = PD+DQ = s

and we deduce

\displaystyle PX = PY = PZ = s.

So {P} is the circumcenter of {\triangle XYZ}. Hence {P} is the isogonal conjugate of {Q}. \Box

The converse of this theorem is also true; given isogonal conjugates {P} and {Q} inside {ABC} we can construct a suitable ellipse. Moreover, it’s worth noting that the lines {AD}, {BE}, {CF} are also concurrent; one proof is to take a projective transformation which sends the ellipse to a circle.

Using this theorem, we can give a “morally correct” solution to the following problem, which is IMO Shortlist 2000, Problem G3.

Problem Let {O} be the circumcenter and {H} the orthocenter of an acute triangle {ABC}. Show that there exist points {D}, {E}, and {F} on sides {BC}, {CA}, and {AB} respectively such that

\displaystyle OD + DH = OE + EH = OF + FH

and the lines {AD}, {BE}, and {CF} are concurrent.

Proof: Because {O} and {H} are isogonal conjugates we can construct an ellipse tangent to the sides at {D}, {E}, {F} from which both conditions follow. \Box

5. Pascal’s Theorem

For more on isogonal conjugates, see e.g. Darij Grinberg. I’ll just leave off with one more nice application of isogonal conjugates, communicated to me by M Kural last August.

Theorem 4 (Pascal) Let {AEBDFC} by a cyclic hexagon, as shown. Suppose {P = \overline{AB} \cap \overline{DE}} {Q = \overline{CD} \cap \overline{FA}}, and {X = \overline{BC} \cap \overline{EF}}. Then points {P}, {X}, {Q} are collinear.


Proof: Notice that {\triangle XEB \sim \triangle XCF}, though the triangles have opposite orientations. Because {\angle BEP = \angle BED = \angle BCD = \angle XCQ}, and so on, the points {P} and {Q} correspond to isogonal conjugates. Hence {\angle EXP = \angle QXF}, which gives the collinearity. \Box

Thanks to R Alweiss and heron1618 for pointing out a few typos, and Daniel Paleka for noticing a careless application of Brianchon’s theorem.

Why do roots come in conjugate pairs?

This is an expanded version of an answer I gave to a question that came up while I was assisting the 2014-2015 WOOT class. It struck me as an unusually good way to motivate higher math using stuff that people notice in high school but for some reason decide to not think about.

In high school precalculus, you’ll often be asked to find the roots of some polynomial with integer coefficients. For instance,

\displaystyle x^3 - x^2 - x - 15 = (x-3)(x^2+2x+5)

has roots {3}, {1+2i}, {-1-2i}. Or as another example,

\displaystyle x^3 - 3x^2 - 2x + 2 = (x+1)(x^2-4x+2)

has roots {-1}, {2 + \sqrt 2}, {2 - \sqrt 2}. You’ll notice that the “weird” roots, like {1 \pm 2i} and {2 \pm \sqrt 2}, are coming up in pairs. In fact, I think precalculus explicitly tells you that the imaginary roots come in conjugate pairs. More generally, it seems like all the roots of the form {a + b \sqrt c} come in “conjugate pairs”. And you can see why.

But a polynomial like

\displaystyle x^3 - 8x + 4

has no rational roots. (The roots of this are approximately {-3.0514}, {0.51730}, {2.5341}.) Or even simpler,

\displaystyle x^3 - 2

has only one real root, {\sqrt[3]{2}}. These roots, even though they are irrational, have no “conjugate” pairs. Or do they?

Let’s try and figure out exactly what’s happening. Let {\alpha} be any complex number. We define the minimal polynomial of {\alpha} to be the monic polynomial {P(x)} such that

  • {P(x)} has rational coefficients, and leading coefficient {1},
  • {P(\alpha) = 0}.
  • The degree of {P} is as small as possible.

For example, {\sqrt 2} has minimal polynomial {x^2-2}. Note that {100x^2 - 200} is also a polynomial of the same degree which has {\sqrt 2} as a root; that’s why we want to require the polynomial to be monic. That’s also why we choose to work in the rational numbers; that way, we can divide by leading coefficients without worrying if we get non-integers.

Why do we care? The point is as follows: suppose we have another polynomial {A(x)} such that {A(\alpha) = 0}. Then we claim that {P(x)} actually divides {A(x)}! That means that all the other roots of {P} will also be roots of {A}.

The proof is by contradiction: if not, by polynomial long division, we can find a quotient and remainder {Q(x)}, {R(x)}such that

\displaystyle A(x) = Q(x) P(x) + R(x)

and {R(x) \not\equiv 0}. Notice that by plugging in {x = \alpha}, we find that {R(\alpha) = 0}. But {\deg R < \deg P}, and {P(x)} was supposed to be the minimal polynomial. That’s impossible!

Let’s look at a more concrete example. Consider {A(x) = x^3-3x^2-2x+2} from the beginning. The minimal polynomial of {2 + \sqrt 2} is {P(x) = x^2 - 4x + 2} (why?). Now we know that if {2 + \sqrt 2} is a root, then {A(x)} is divisible by {P(x)}. And that’s how we know that if {2 + \sqrt 2} is a root of {A}, so must {2 - \sqrt 2}.

As another example, the minimal polynomial of {\sqrt[3]{2}} is {x^3-2}. So {\sqrt[3]{2}} actually has two conjugates, namely, {\alpha = \sqrt[3]{2} \left( \cos 120^\circ + i \sin 120^\circ \right)} and {\beta = \sqrt[3]{2} \left( \cos 240^\circ + i \sin 240^\circ \right)}. Thus any polynomial which vanishes at {\sqrt[3]{2}} also has {\alpha} and {\beta} as roots!

You can generalize this by replacing {\mathbb Q} with any field and all of this still works. One central idea of Galois theory is that these “conjugates” all “look the same” as far as {\mathbb Q} can tell.

As another aside: does the minimal polynomial exist for every {\alpha}? It turns out the answer is no, and the numbers for which there is no minimal polynomial are called the transcendental numbers.