What can College Scorecard data tell us about Harvard graduates?

Who is in the data, generalizability, and causality

Harvard Statistics in The New York Times

On October 7, 2026, The New York Times published an interactive guide to what graduates of each college program earn, and Harvard Statistics makes an appearance. The guide uses College Scorecard data from the U.S. Department of Education. In this handout we look at the same data and ask: what can these numbers tell us, and what can’t they? The following plot is NOT from the NYT.

Strip plot of median earnings four years after graduation for every U.S. bachelor's program in the fields of Harvard's reported concentrations, with Harvard highlighted in crimson.

What is in the College Scorecard data?

Everything in this section comes from the College Scorecard documentation and data files (see Sources at the end).

  • Only federal aid recipients. Earnings are reported only for graduates who received federal financial aid (the documentation calls it “Title IV” aid). The documentation warns that because not all graduates received this aid, we “should not assume that Scorecard data values are representative of all students.”
  • How many Harvard students is that? In the years these graduates were at Harvard (2013-14 to 2018-19), the Scorecard data show that about 11-13% of Harvard undergraduates received a Pell Grant each year and about 2-5% received a federal loan.
  • Where the earnings come from. Federal aid records are linked to IRS tax records: wages from W-2 forms plus self-employment earnings.
  • Who is left out. Graduates who, during the year their earnings were measured, were enrolled in school, did not work, or had loan deferments for military service. Graduates who died before the end of that year are also left out.
  • Which graduates and when. Two graduating classes are combined (2017-18 and 2018-19). Their earnings were measured in 2022 and 2023, about four years after graduation, and are reported in 2024 dollars.
  • Privacy. To protect graduates’ privacy, the Scorecard adds a small amount of random error to the earnings numbers on purpose. Statisticians call this added random error noise. Medians that became too unreliable are hidden (“privacy suppressed”), and the exact rules are not public. Harvard concentrations with hidden medians are not shown in the plot.
  • Small numbers. The Harvard Statistics median is based on 17 graduates.
  • Most statistics programs are not reported. In the Scorecard data, 177 U.S. colleges awarded at least one statistics bachelor’s degree in 2022-23, but only 50 statistics programs have a reported 4-year median. So when we say Harvard Statistics is “#1”, we mean #1 among those 50. The statistics programs that awarded the most degrees but have no reported median include New York University, University of Pennsylvania, Yale University, Ohio State University-Main Campus, and University of Arizona.

Questions

Multiple choice

  1. Your roommate says: “Half of all Harvard Statistics graduates earn more than the crimson dot’s value four years after graduation.” What is the best response?

    1. Correct, because the median splits the data in half
    2. Incorrect. The median describes federal aid recipients who were working and not enrolled, not all Statistics graduates
    3. Incorrect, because the median is affected by outliers
    4. Correct, because the data come from the IRS, which covers everyone

  2. The plot’s title says “Every reported Harvard concentration is in the top half.” Which conclusion is justified?

    1. Every Harvard concentration is in the top half of its field
    2. Every Harvard concentration whose median was published (not hidden for privacy) is in the top half of its field in terms of earnings
    3. Harvard has only 11 concentrations
    4. Concentrations that are not shown have below-median earnings
    5. None of the above

    If you chose e, rewrite option b so that it is justified.

Short answer

  1. Small groups. The Harvard Statistics median comes from 17 graduates. Imagine a different group of 17 Harvard Statistics graduates. Their median would probably not be exactly the same. Statisticians call this sampling variability: a number we calculate from one group of people changes from group to group just because of which people happened to be in it.

    Which do you think is the bigger problem for saying something about all Harvard Statistics graduates: sampling variability, or the fact that only federal aid recipients who were working and not enrolled are counted? Explain in 2-3 sentences.

  2. Noise. The Scorecard adds a small amount of random error (noise) to the earnings numbers on purpose to protect privacy. Why might it be reasonable to accept slightly less accurate numbers in exchange for privacy? Give one reason.

  3. The missing programs. Harvard Statistics is #1 among the 50 statistics programs with a reported median. Programs such as New York University, University of Pennsylvania, and Yale University awarded statistics degrees but have no reported median.

    1. Can we say Harvard Statistics has the highest median of all U.S. statistics programs? Why or why not?
    2. A median is hidden when the Scorecard judges it too unreliable. Would you expect a program with few graduates in the data or many graduates in the data to be more likely to be hidden? Explain.
    3. If the hidden programs differ from the reported ones (for example, in size), what does that mean for treating the 50 reported programs as a picture of all statistics programs?

  4. Causality. A first-year says: “If I switch concentrations from English to Statistics, I’ll earn about $180,000 more.” Students choose their own concentrations. Explain why the plot cannot tell us what would happen if this student switched.

  5. Confounding. Name one variable that might affect both which concentration a student chooses and how much they earn later.

  6. Generalizing over time. These graduates finished college in 2018 and 2019. Give one reason the Statistics median may not describe what your class will earn four years after graduation.

  7. Write the headline. A friend posts: “Harvard Statistics grads make $231K four years out, #1 in the country!” Rewrite the post in one sentence so that it is accurate about who the number describes and which programs Harvard is compared with.

Multiple choice:

  1. b
    1. Option b goes too far in two ways. First, Harvard is compared only with programs in the same field that have a reported median, not with all programs in its field. Second, the median describes only graduates who received federal aid and were working and not enrolled, not the earnings of the concentration’s graduates in general. A justified version: “Among U.S. programs in the same field with a reported median, every reported Harvard concentration’s median (for federal aid recipients who were working and not enrolled) is in the top half.”

Short answer (sample answers):

  1. Who is counted is the bigger problem. Even with no sampling variability, the number would describe federal aid recipients who were working and not enrolled, not all Statistics graduates. A larger group of the same kind of graduates would not fix that.
  2. Earnings are personal information. With small groups, such as 17 graduates, exact numbers could reveal something about individual people.
    1. No. We don’t know the medians of the hidden programs, so any of them could be higher or lower. Harvard is #1 only among programs with a reported median. (b) Programs with few graduates in the data: a median from a small group is less reliable, especially after noise is added. (c) The reported programs may not represent all statistics programs, so rankings and comparisons apply only to the reported ones.
  4. Students who choose Statistics may differ from students who choose English in interests, career plans, and the jobs they want. The plot compares different people; it does not show the same person under both choices.
  5. For example, career plans made before choosing a concentration (such as wanting to work in tech or finance).
  6. These are graduates from 2018 and 2019. Your class will graduate into a different job market, and you may differ from the federal aid recipients in the data.
  7. For example: “Among Harvard Statistics graduates of 2018 and 2019 who received federal aid and were working (not in school) four years later, the median was about $231K, based on 17 people, the highest of the 50 U.S. statistics programs with reported earnings.”