U.S. students in different ethnic groups have very different average scores on the PISA 2012 mathematics and reading tests, with Blacks and Hispanics showing negative gaps relative to White students and Asians showing a positive gap. I investigate whether a student’s family characteristics or the school attended can explain these differences. I find that Hispanic parents’ low average education explains the largest share of the Hispanic achievement gap. In contrast, most of the larger negative gap for Blacks and the positive gap for Asians cannot be explained by family characteristics or the school they attend. Attendance at “bad” schools explains relatively little of the negative gaps, but Black students’ mathematics scores are substantially lower when they compose more than 50% of the class, which is not the case for Hispanic students. This evidence suggests that ethnic group culture is an important cause of Black and Asian student achievement gaps.